The timing of the cognitive cycle

Content-state dimensions characterize different types of neuronal markers of consciousness

Identifying the neuronal markers of consciousness is key to supporting the different scientific theories of consciousness. Neuronal markers of consciousness can be defined to reflect either the brain signatures underlying specific conscious content or those supporting different states of consciousness, two aspects traditionally studied separately. In this paper, we introduce a framework to characterize markers according to their dynamics in both the "state" and "content" dimensions. The 2D space is defined by the marker's capacity to distinguish the conscious states from non-conscious states (on the x-axis) and the content (e.g. perceived versus unperceived or different levels of cognitive processing on the y-axis). According to the sign of the x- and y-axis, markers are separated into four quadrants in terms of how they distinguish the state and content dimensions. We implement the framework using three types of electroencephalography markers: markers of connectivity, markers of complexity, and spectral summaries. The neuronal markers of state are represented by the level of consciousness in (i) healthy participants during a nap and (ii) patients with disorders of consciousness. On the other hand, the neuronal markers of content are represented by (i) the conscious content in healthy participants' perception task using a visual awareness paradigm and (ii) conscious processing of hierarchical regularities using an auditory local-global paradigm. In both cases, we see separate clusters of markers with correlated and anticorrelated dynamics, shedding light on the complex relationship between the state and content of consciousness and emphasizing the importance of considering them simultaneously. This work presents an innovative framework for studying consciousness by examining neuronal markers in a 2D space, providing a valuable resource for future research, with potential applications using diverse experimental paradigms, neural recording techniques, and modeling investigations.

The emergence of enhanced intelligence in a brain-inspired cognitive architecture

The Causal Cognitive Architecture is a brain-inspired cognitive architecture developed from the hypothesis that the navigation circuits in the ancestors of mammals duplicated to eventually form the neocortex. Thus, millions of neocortical minicolumns are functionally modeled in the architecture as millions of "navigation maps." An investigation of a cognitive architecture based on these navigation maps has previously shown that modest changes in the architecture allow the ready emergence of human cognitive abilities such as grounded, full causal decision-making, full analogical reasoning, and near-full compositional language abilities. In this study, additional biologically plausible modest changes to the architecture are considered and show the emergence of super-human planning abilities. The architecture should be considered as a viable alternative pathway toward the development of more advanced artificial intelligence, as well as to give insight into the emergence of natural human intelligence.

EEG Phase Can Be Predicted with Similar Accuracy across Cognitive States after Accounting for Power and Signal-to-Noise Ratio

EEG phase is increasingly used in cognitive neuroscience, brain-computer interfaces, and closed-loop stimulation devices. However, it is unknown how accurate EEG phase prediction is across cognitive states. We determined the EEG phase prediction accuracy of parieto-occipital alpha waves across rest and task states in 484 participants over 11 public datasets. We were able to track EEG phase accurately across various cognitive conditions and datasets, especially during periods of high instantaneous alpha power and signal-to-noise ratio (SNR). Although resting states generally have higher accuracies than task states, absolute accuracy differences were small, with most of these differences attributable to EEG power and SNR. These results suggest that experiments and technologies using EEG phase should focus more on minimizing external noise and waiting for periods of high power rather than inducing a particular cognitive state.

Prognostication after cardiac arrest: how EEG and evoked potentials may improve the challenge

About 80% of patients resuscitated from CA are comatose at ICU admission and nearly 50% of survivors are still unawake at 72 h. Predicting neurological outcome of these patients is important to provide correct information to patient's relatives, avoid disproportionate care in patients with irreversible hypoxic-ischemic brain injury (HIBI) and inappropriate withdrawal of care in patients with a possible favorable neurological recovery. ERC/ESICM 2021 algorithm allows a classification as "poor outcome likely" in 32%, the outcome remaining "indeterminate" in 68%. The crucial question is to know how we could improve the assessment of both unfavorable but also favorable outcome prediction. Neurophysiological tests, i.e., electroencephalography (EEG) and evoked-potentials (EPs) are a non-invasive bedside investigations. The EEG is the record of brain electrical fields, characterized by a high temporal resolution but a low spatial resolution. EEG is largely available, and represented the most widely tool use in recent survey examining current neuro-prognostication practices. The severity of HIBI is correlated with the predominant frequency and background continuity of EEG leading to "highly malignant" patterns as suppression or burst suppression in the most severe HIBI. EPs differ from EEG signals as they are stimulus induced and represent the summated activities of large populations of neurons firing in synchrony, requiring the average of numerous stimulations. Different EPs (i.e., somato sensory EPs (SSEPs), brainstem auditory EPs (BAEPs), middle latency auditory EPs (MLAEPs) and long latency event-related potentials (ERPs) with mismatch negativity (MMN) and P300 responses) can be assessed in ICU, with different brain generators and prognostic values. In the present review, we summarize EEG and EPs signal generators, recording modalities, interpretation and prognostic values of these different neurophysiological tools. Finally, we assess the perspective for futures neurophysiological investigations, aiming to reduce prognostic uncertainty in comatose and disorders of consciousness (DoC) patients after CA.

Integrated world modeling theory expanded: Implications for the future of consciousness

Integrated world modeling theory (IWMT) is a synthetic theory of consciousness that uses the free energy principle and active inference (FEP-AI) framework to combine insights from integrated information theory (IIT) and global neuronal workspace theory (GNWT). Here, I first review philosophical principles and neural systems contributing to IWMT's integrative perspective. I then go on to describe predictive processing models of brains and their connections to machine learning architectures, with particular emphasis on autoencoders (perceptual and active inference), turbo-codes (establishment of shared latent spaces for multi-modal integration and inferential synergy), and graph neural networks (spatial and somatic modeling and control). Future directions for IIT and GNWT are considered by exploring ways in which modules and workspaces may be evaluated as both complexes of integrated information and arenas for iterated Bayesian model selection. Based on these considerations, I suggest novel ways in which integrated information might be estimated using concepts from probabilistic graphical models, flow networks, and game theory. Mechanistic and computational principles are also considered with respect to the ongoing debate between IIT and GNWT regarding the physical substrates of different kinds of conscious and unconscious phenomena. I further explore how these ideas might relate to the "Bayesian blur problem," or how it is that a seemingly discrete experience can be generated from probabilistic modeling, with some consideration of analogies from quantum mechanics as potentially revealing different varieties of inferential dynamics. I go on to describe potential means of addressing critiques of causal structure theories based on network unfolding, and the seeming absurdity of conscious expander graphs (without cybernetic symbol grounding). Finally, I discuss future directions for work centered on attentional selection and the evolutionary origins of consciousness as facilitated "unlimited associative learning." While not quite solving the Hard problem, this article expands on IWMT as a unifying model of consciousness and the potential future evolution of minds.

Spontaneous neuronal avalanches as a correlate of access consciousness

Decades of research have advanced our understanding of the biophysical mechanisms underlying consciousness. However, an overarching framework bridging between models of consciousness and the large-scale organization of spontaneous brain activity is still missing. Based on the observation that spontaneous brain activity dynamically switches between epochs of segregation and large-scale integration of information, we hypothesize a brain-state dependence of conscious access, whereby the presence of either segregated or integrated states marks distinct modes of information processing. We first review influential works on the neuronal correlates of consciousness, spontaneous resting-state brain activity and dynamical system theory. Then, we propose a test experiment to validate our hypothesis that conscious access occurs in aperiodic cycles, alternating windows where new incoming information is collected but not experienced, to punctuated short-lived integration events, where conscious access to previously collected content occurs. In particular, we suggest that the integration events correspond to neuronal avalanches, which are collective bursts of neuronal activity ubiquitously observed in electrophysiological recordings. If confirmed, the proposed framework would link the physics of spontaneous cortical dynamics, to the concept of ignition within the global neuronal workspace theory, whereby conscious access manifest itself as a burst of neuronal activity.

Navigation Map-Based Artificial Intelligence

A biologically inspired cognitive architecture is described which uses navigation maps (i.e., spatial locations of objects) as its main data elements. The navigation maps are also used to represent higher-level concepts as well as to direct operations to perform on other navigation maps. Incoming sensory information is mapped to local sensory navigation maps which then are in turn matched with the closest multisensory maps, and then mapped onto a best-matched multisensory navigation map. Enhancements of the biologically inspired feedback pathways allow the intermediate results of operations performed on the best-matched multisensory navigation map to be fed back, temporarily stored, and re-processed in the next cognitive cycle. This allows the exploration and generation of cause-and-effect behavior. In the re-processing of these intermediate results, navigation maps can, by core analogical mechanisms, lead to other navigation maps which offer an improved solution to many routine problems the architecture is exposed to. Given that the architecture is brain-inspired, analogical processing may also form a key mechanism in the human brain, consistent with psychological evidence. Similarly, for conventional artificial intelligence systems, analogical processing as a core mechanism may possibly allow enhanced performance.

Embodied Intelligence: Smooth Coping in the Learning Intelligent Decision Agent Cognitive Architecture

Much of our everyday, embodied action comes in the form of smooth coping. Smooth coping is skillful action that has become habituated and ingrained, generally placing less stress on cognitive load than considered and deliberative thought and action. When performed with skill and expertise, walking, driving, skiing, musical performances, and short-order cooking are all examples of the phenomenon. Smooth coping is characterized by its rapidity and relative lack of reflection, both being hallmarks of automatization. Deliberative and reflective actions provide the contrast case. In Dreyfus' classic view, smooth coping is "mindless" absorption into action, being in the flow, and any reflective thought will only interrupt this flow. Building on the pragmatist account of Dewey, others, such as Sutton, Montero, and Gallagher, insist on the intelligent flexibility built into smooth coping, suggesting that it is not equivalent to automatization. We seek to answer two complementary challenges in this article. First, how might we model smooth coping in autonomous agents (natural or artificial) at fine granularity? Second, we use this model of smooth coping to show how we might implement smooth coping in artificial intelligent agents. We develop a conceptual model of smooth coping in LIDA (Learning Intelligent Decision Agent). LIDA is an embodied cognitive architecture implementing the global workspace theory of consciousness, among other psychological theories. LIDA's implementation of consciousness enables us to account for the phenomenology of smooth coping, something that few cognitive architectures would be able to do. Through the fine granular analysis of LIDA, we argue that smooth coping is a sequence of automatized actions intermittently interspersed with consciously mediated action selection, supplemented by dorsal stream processes. In other words, non-conscious, automatized actions (whether learned or innate) often require occasional bursts of conscious cognition to achieve the skillful and flexible adjustments of smooth coping. In addition, never-conscious dorsal stream information and associated sensorimotor processes provide further online adjustments during smooth coping. To achieve smooth coping in LIDA we introduce a new module to the LIDA cognitive architecture the Automatized Action Selection sub-module. Our complex model of smooth coping borrows notions of "embodied intelligence" from enactivism and augments these by allowing representations and more detailed mechanisms of conscious control. We explore several extended examples of smooth coping, starting from basic activities like walking and scaling up to more complex tasks like driving and short-order cooking.

An Introduction to Musical Interactions

The article presents a contextual survey of eight contributions in the special issue Musical Interactions (Volume I) in Multimodal Technologies and Interaction. The presentation includes (1) a critical examination of what it means to be musical, to devise the concept of music proper to MTI as well as multicultural proximity, and (2) a conceptual framework for instrumentation, design, and assessment of musical interaction research through five enabling dimensions: Affordance; Design Alignment; Adaptive Learning; Second-Order Feedback; Temporal Integration. Each dimension is discussed and applied in the survey. The results demonstrate how the framework provides an interdisciplinary scope required for musical interaction, and how this approach may offer a coherent way to describe and assess approaches to research and design as well as implementations of interactive musical systems. Musical interaction stipulates musical liveness for experiencing both music and technologies. While music may be considered ontologically incomplete without a listener, musical interaction is defined as ontological completion of a state of music and listening through a listener’s active engagement with musical resources in multimodal information flow.

Evoked and Event-Related Potentials as Biomarkers of Consciousness State and Recovery

SUMMARY

The definition of consciousness has been the subject of great interest for many scientists and philosophers. To better understand how evoked potentials may be identified as biomarkers of consciousness and recovery, the different theoretical models sustaining neural correlates of consciousness are reviewed. A multimodal approach can help to better predict clinical outcome in patients presenting with disorders of consciousness. Evoked potentials are inexpensive and easy-to-implement bedside examination techniques. Evoked potentials are an integral part of prognostic evaluation, particularly in cases of cognitive motor dissociation. Prognostic criteria are well established in postanoxic disorders of consciousness, especially postcardiac arrest but are less well determined in other etiologies. In the early examination, bilateral absence of N20 in disorder of consciousness patients is strongly associated with unfavorable outcome (i.e., death or unresponsive wakefulness syndrome) especially in postanoxic etiologies. This predictive value is lower in other etiologies and probably also in children. Both N20 and mismatch negativity are proven outcome predictors for acute coma. Many studies have shown that mismatch negativity and P3a are characterized by a high prognostic value for awakening, but some patients presenting unresponsive wakefulness syndrome also process a P3a. The presence of long-latency event-related potential components in response to stimuli is indicative of a better recovery. All neurophysiological data must be integrated within a multimodal approach combining repeated clinical evaluation, neuroimaging, functional imaging, biology, and neurophysiology combining passive and active paradigms.

Evolutionary Advantages of Stimulus-Driven EEG Phase Transitions in the Upper Cortical Layers

Spatio-temporal brain activity monitored by EEG recordings in humans and other mammals has identified beta/gamma oscillations (20-80 Hz), which are self-organized into spatio-temporal structures recurring at theta/alpha rates (4-12 Hz). These structures have statistically significant correlations with sensory stimuli and reinforcement contingencies perceived by the subject. The repeated collapse of self-organized structures at theta/alpha rates generates laterally propagating phase gradients (phase cones), ignited at some specific location of the cortical sheet. Phase cones have been interpreted as neural signatures of transient perceptual experiences according to the cinematic theory of brain dynamics. The rapid expansion of essentially isotropic phase cones is consistent with the propagation of perceptual broadcasts postulated by Global Workspace Theory (GWT). What is the evolutionary advantage of brains operating with repeatedly collapsing dynamics? This question is answered using thermodynamic concepts. According to neuropercolation theory, waking brains are described as non-equilibrium thermodynamic systems operating at the edge of criticality, undergoing repeated phase transitions. This work analyzes the role of long-range axonal connections and metabolic processes in the regulation of critical brain dynamics. Historically, the near 10 Hz domain has been associated with conscious sensory integration, cortical "ignitions" linked to conscious visual perception, and conscious experiences. We can therefore combine a very large body of experimental evidence and theory, including graph theory, neuropercolation, and GWT. This cortical operating style may optimize a tradeoff between rapid adaptation to novelty vs. stable and widespread self-organization, therefore resulting in significant Darwinian benefits.

Perceptual and Semantic Processing in Cognitive Robots

The challenge in human–robot interaction is to build an agent that can act upon human implicit statements, where the agent is instructed to execute tasks without explicit utterance. Understanding what to do under such scenarios requires the agent to have the capability to process object grounding and affordance learning from acquired knowledge. Affordance has been the driving force for agents to construct relationships between objects, their effects, and actions, whereas grounding is effective in the understanding of spatial maps of objects present in the environment. The main contribution of this paper is to propose a methodology for the extension of object affordance and grounding, the Bloom-based cognitive cycle, and the formulation of perceptual semantics for the context-based human–robot interaction. In this study, we implemented YOLOv3 to formulate visual perception and LSTM to identify the level of the cognitive cycle, as cognitive processes synchronized in the cognitive cycle. In addition, we used semantic networks and conceptual graphs as a method to represent knowledge in various dimensions related to the cognitive cycle. The visual perception showed average precision of 0.78, an average recall of 0.87, and an average F1 score of 0.80, indicating an improvement in the generation of semantic networks and conceptual graphs. The similarity index used for the lingual and visual association showed promising results and improves the overall experience of human–robot interaction.

Time and time again: a multi-scale hierarchical framework for time-consciousness and timing of cognition

Temporality and the feeling of ‘now’ is a fundamental property of consciousness. Different conceptualizations of time-consciousness have argued that both the content of our experiences and the representations of those experiences evolve in time, or neither have temporal extension, or only content does. Accounting for these different positions, we propose a nested hierarchical model of multiple timescales that accounts for findings on timing of cognition and phenomenology of temporal experience. This framework hierarchically combines the three major philosophical positions on time-consciousness (i.e. cinematic, extensional and retentional) and presents a common basis for temporal experience. We detail the properties of these hierarchical levels and speculate how they could coexist mechanistically. We also place several findings on timing and temporal experience at different levels in this hierarchy and show how they can be brought together. Finally, the framework is used to derive novel predictions for both timing of our experiences and time perception. The theoretical framework offers a novel dynamic space that can bring together sub-fields of cognitive science like perception, attention, action and consciousness research in understanding and describing our experiences both in and of time.

The Radically Embodied Conscious Cybernetic Bayesian Brain: From Free Energy to Free Will and Back Again

Drawing from both enactivist and cognitivist perspectives on mind, I propose that explaining teleological phenomena may require reappraising both "Cartesian theaters" and mental homunculi in terms of embodied self-models (ESMs), understood as body maps with agentic properties, functioning as predictive-memory systems and cybernetic controllers. Quasi-homuncular ESMs are suggested to constitute a major organizing principle for neural architectures due to their initial and ongoing significance for solutions to inference problems in cognitive (and affective) development. Embodied experiences provide foundational lessons in learning curriculums in which agents explore increasingly challenging problem spaces, so answering an unresolved question in Bayesian cognitive science: what are biologically plausible mechanisms for equipping learners with sufficiently powerful inductive biases to adequately constrain inference spaces? Drawing on models from neurophysiology, psychology, and developmental robotics, I describe how embodiment provides fundamental sources of empirical priors (as reliably learnable posterior expectations). If ESMs play this kind of foundational role in cognitive development, then bidirectional linkages will be found between all sensory modalities and frontal-parietal control hierarchies, so infusing all senses with somatic-motoric properties, thereby structuring all perception by relevant affordances, so solving frame problems for embodied agents. Drawing upon the Free Energy Principle and Active Inference framework, I describe a particular mechanism for intentional action selection via consciously imagined (and explicitly represented) goal realization, where contrasts between desired and present states influence ongoing policy selection via predictive coding mechanisms and backward-chained imaginings (as self-realizing predictions). This embodied developmental legacy suggests a mechanism by which imaginings can be intentionally shaped by (internalized) partially-expressed motor acts, so providing means of agentic control for attention, working memory, imagination, and behavior. I further describe the nature(s) of mental causation and self-control, and also provide an account of readiness potentials in Libet paradigms wherein conscious intentions shape causal streams leading to enaction. Finally, I provide neurophenomenological handlings of prototypical qualia including pleasure, pain, and desire in terms of self-annihilating free energy gradients via quasi-synesthetic interoceptive active inference. In brief, this manuscript is intended to illustrate how radically embodied minds may create foundations for intelligence (as capacity for learning and inference), consciousness (as somatically-grounded self-world modeling), and will (as deployment of predictive models for enacting valued goals).

An Integrated World Modeling Theory (IWMT) of Consciousness: Combining Integrated Information and Global Neuronal Workspace Theories With the Free Energy Principle and Active Inference Framework; Toward Solving the Hard Problem and Characterizing Agentic Causation

The Free Energy Principle and Active Inference Framework (FEP-AI) begins with the understanding that persisting systems must regulate environmental exchanges and prevent entropic accumulation. In FEP-AI, minds and brains are predictive controllers for autonomous systems, where action-driven perception is realized as probabilistic inference. Integrated Information Theory (IIT) begins with considering the preconditions for a system to intrinsically exist, as well as axioms regarding the nature of consciousness. IIT has produced controversy because of its surprising entailments: quasi-panpsychism; subjectivity without referents or dynamics; and the possibility of fully-intelligent-yet-unconscious brain simulations. Here, I describe how these controversies might be resolved by integrating IIT with FEP-AI, where integrated information only entails consciousness for systems with perspectival reference frames capable of generating models with spatial, temporal, and causal coherence for self and world. Without that connection with external reality, systems could have arbitrarily high amounts of integrated information, but nonetheless would not entail subjective experience. I further describe how an integration of these frameworks may contribute to their evolution as unified systems theories and models of emergent causation. Then, inspired by both Global Neuronal Workspace Theory (GNWT) and the Harmonic Brain Modes framework, I describe how streams of consciousness may emerge as an evolving generation of sensorimotor predictions, with the precise composition of experiences depending on the integration abilities of synchronous complexes as self-organizing harmonic modes (SOHMs). These integrating dynamics may be particularly likely to occur via richly connected subnetworks affording body-centric sources of phenomenal binding and executive control. Along these connectivity backbones, SOHMs are proposed to implement turbo coding via loopy message-passing over predictive (autoencoding) networks, thus generating maximum a posteriori estimates as coherent vectors governing neural evolution, with alpha frequencies generating basic awareness, and cross-frequency phase-coupling within theta frequencies for access consciousness and volitional control. These dynamic cores of integrated information also function as global workspaces, centered on posterior cortices, but capable of being entrained with frontal cortices and interoceptive hierarchies, thus affording agentic causation. Integrated World Modeling Theory (IWMT) represents a synthetic approach to understanding minds that reveals compatibility between leading theories of consciousness, thus enabling inferential synergy.

A real-time marker of object-based attention in the human brain. A possible component of a "gate-keeping mechanism" performing late attentional selection in the Ventro-Lateral Prefrontal Cortex

The decision to process an incoming stimulus attentively - and to trigger a follow-up cascade of high-level processes - is strategic for the human brain as it becomes transiently unavailable to subsequent stimulus processing. In this study, we set to identify brain networks that carry out such evaluations. We therefore assessed the time-course of neural responses with intracerebral EEG in human patients during an attentional reading task, contrasting to-be-attended vs. to-be-ignored items. We measured High-Frequency Activity [50-150 ​Hz] as a proxy of population-level spiking activity and we identified a crucial component of a Gate-Keeping Mechanism bilateral in the mid-Ventro-Lateral Prefrontal Cortex (VLPFC), at the interplay of the Ventral and Dorsal Attention Networks, that selectively reacts before domain specialized cortical regions that engage in full stimulus analysis according to task demands.

BLAST: A short computerized test to measure the ability to stay on task. Normative behavioral data and detailed cortical dynamics

This article provides an exhaustive description of a new short computerized test to assess on a second-to-second basis the ability of individuals to « stay on task », that is, to apply selectively and repeatedly task-relevant cognitive processes. The task (Bron/Lyon Attention Stability Test, or BLAST) lasts around 1 min, and measures repeatedly the time to find a target letter in a two-by-two letter array, with an update of all letters every new trial across thirty trials. Several innovative psychometric measures of attention stability are proposed based on the instantaneous fluctuations of reaction times throughout the task, and normative data stratified over a wide range of age are provided by a large (>6000) dataset of participants aged 8 to 70. We also detail the large-scale brain dynamics supporting the task from an in-depth study of 32 participants with direct electrophysiological cortical recordings (intracranial EEG) to prove that BLAST involves critically large-scale executive attention networks, with a marked activation of the dorsal attention network and a deactivation of the default-mode network. Accordingly, we show that BLAST performance correlates with scores established by ADHD-questionnaires.

Interactive Sonification Exploring Emergent Behavior Applying Models for Biological Information and Listening

Sonification is an open-ended design task to construct sound informing a listener of data. Understanding application context is critical for shaping design requirements for data translation into sound. Sonification requires methodology to maintain reproducibility when data sources exhibit non-linear properties of self-organization and emergent behavior. This research formalizes interactive sonification in an extensible model to support reproducibility when data exhibits emergent behavior. In the absence of sonification theory, extensibility demonstrates relevant methods across case studies. The interactive sonification framework foregrounds three factors: reproducible system implementation for generating sonification; interactive mechanisms enhancing a listener's multisensory observations; and reproducible data from models that characterize emergent behavior. Supramodal attention research suggests interactive exploration with auditory feedback can generate context for recognizing irregular patterns and transient dynamics. The sonification framework provides circular causality as a signal pathway for modeling a listener interacting with emergent behavior. The extensible sonification model adopts a data acquisition pathway to formalize functional symmetry across three subsystems: Experimental Data Source, Sound Generation, and Guided Exploration. To differentiate time criticality and dimensionality of emerging dynamics, tuning functions are applied between subsystems to maintain scale and symmetry of concurrent processes and temporal dynamics. Tuning functions accommodate sonification design strategies that yield order parameter values to render emerging patterns discoverable as well as rehearsable, to reproduce desired instances for clinical listeners. Case studies are implemented with two computational models, Chua's circuit and Swarm Chemistry social agent simulation, generating data in real-time that exhibits emergent behavior. Heuristic Listening is introduced as an informal model of a listener's clinical attention to data sonification through multisensory interaction in a context of structured inquiry. Three methods are introduced to assess the proposed sonification framework: Listening Scenario classification, data flow Attunement, and Sonification Design Patterns to classify sound control. Case study implementations are assessed against these methods comparing levels of abstraction between experimental data and sound generation. Outcomes demonstrate the framework performance as a reference model for representing experimental implementations, also for identifying common sonification structures having different experimental implementations, identifying common functions implemented in different subsystems, and comparing impact of affordances across multiple implementations of listening scenarios.

Breeding novel solutions in the brain: a model of Darwinian neurodynamics

Background: The fact that surplus connections and neurons are pruned during development is well established. We complement this selectionist picture by a proof-of-principle model of evolutionary search in the brain, that accounts for new variations in theory space. We present a model for Darwinian evolutionary search for candidate solutions in the brain.

Methods: We combine known components of the brain – recurrent neural networks (acting as attractors), the action selection loop and implicit working memory – to provide the appropriate Darwinian architecture. We employ a population of attractor networks with palimpsest memory. The action selection loop is employed with winners-share-all dynamics to select for candidate solutions that are transiently stored in implicit working memory.

Results: We document two processes: selection of stored solutions and evolutionary search for novel solutions. During the replication of candidate solutions attractor networks occasionally produce recombinant patterns, increasing variation on which selection can act. Combinatorial search acts on multiplying units (activity patterns) with hereditary variation and novel variants appear due to (i) noisy recall of patterns from the attractor networks, (ii) noise during transmission of candidate solutions as messages between networks, and, (iii) spontaneously generated, untrained patterns in spurious attractors.

Conclusions: Attractor dynamics of recurrent neural networks can be used to model Darwinian search. The proposed architecture can be used for fast search among stored solutions (by selection) and for evolutionary search when novel candidate solutions are generated in successive iterations. Since all the suggested components are present in advanced nervous systems, we hypothesize that the brain could implement a truly evolutionary combinatorial search system, capable of generating novel variants.

Breeding novel solutions in the brain: a model of Darwinian neurodynamics

Background: The fact that surplus connections and neurons are pruned during development is well established. We complement this selectionist picture by a proof-of-principle model of evolutionary search in the brain, that accounts for new variations in theory space. We present a model for Darwinian evolutionary search for candidate solutions in the brain. Methods: We combine known components of the brain - recurrent neural networks (acting as attractors), the action selection loop and implicit working memory - to provide the appropriate Darwinian architecture. We employ a population of attractor networks with palimpsest memory. The action selection loop is employed with winners-share-all dynamics to select for candidate solutions that are transiently stored in implicit working memory. Results: We document two processes: selection of stored solutions and evolutionary search for novel solutions. During the replication of candidate solutions attractor networks occasionally produce recombinant patterns, increasing variation on which selection can act. Combinatorial search acts on multiplying units (activity patterns) with hereditary variation and novel variants appear due to (i) noisy recall of patterns from the attractor networks, (ii) noise during transmission of candidate solutions as messages between networks, and, (iii) spontaneously generated, untrained patterns in spurious attractors. Conclusions: Attractor dynamics of recurrent neural networks can be used to model Darwinian search. The proposed architecture can be used for fast search among stored solutions (by selection) and for evolutionary search when novel candidate solutions are generated in successive iterations. Since all the suggested components are present in advanced nervous systems, we hypothesize that the brain could implement a truly evolutionary combinatorial search system, capable of generating novel variants.

A Cognitive Control-Inspired Approach to Object Tracking

Under a tracking framework, the definition of the target state is the basic step for automatic understanding of dynamic scenes. More specifically, far object tracking raises challenges related to the potentially abrupt size changes of the targets as they approach the sensor. If not handled, size changes can introduce heavy issues in data association and position estimation. This is why adaptability and self-awareness of a tracking module are desirable features. The paradigm of cognitive dynamic systems (CDSs) can provide a framework under which a continuously learning cognitive module can be designed. In particular, CDS theory describes a basic vocabulary of components that can be used as the founding blocks of a module capable to learn behavioral rules from continuous active interactions with the environment. This quality is the fundamental to deal with dynamic situations. In this paper we propose a general CDS-based approach to tracking. We show that such a CDS-inspired design can lead to the self-adaptability of a Bayesian tracker in fusing heterogeneous object features, overcoming size change issues. The experimental results on infrared sequences show how the proposed framework is able to outperform other existing far object tracking methods.

The cerebellum for jocks and nerds alike

Historically the cerebellum has been implicated in the control of movement. However, the cerebellum's role in non-motor functions, including cognitive and emotional processes, has also received increasing attention. Starting from the premise that the uniform architecture of the cerebellum underlies a common mode of information processing, this review examines recent electrophysiological findings on the motor signals encoded in the cerebellar cortex and then relates these signals to observations in the non-motor domain. Simple spike firing of individual Purkinje cells encodes performance errors, both predicting upcoming errors as well as providing feedback about those errors. Further, this dual temporal encoding of prediction and feedback involves a change in the sign of the simple spike modulation. Therefore, Purkinje cell simple spike firing both predicts and responds to feedback about a specific parameter, consistent with computing sensory prediction errors in which the predictions about the consequences of a motor command are compared with the feedback resulting from the motor command execution. These new findings are in contrast with the historical view that complex spikes encode errors. Evaluation of the kinematic coding in the simple spike discharge shows the same dual temporal encoding, suggesting this is a common mode of signal processing in the cerebellar cortex. Decoding analyses show the considerable accuracy of the predictions provided by Purkinje cells across a range of times. Further, individual Purkinje cells encode linearly and independently a multitude of signals, both kinematic and performance errors. Therefore, the cerebellar cortex's capacity to make associations across different sensory, motor and non-motor signals is large. The results from studying how Purkinje cells encode movement signals suggest that the cerebellar cortex circuitry can support associative learning, sequencing, working memory, and forward internal models in non-motor domains.

Microcognitive science: bridging experiential and neuronal microdynamics

Neurophenomenology, as an attempt to combine and mutually enlighten neural and experiential descriptions of cognitive processes, has met practical difficulties which have limited its implementation into actual research projects. The main difficulty seems to be the disparity of the levels of description: while neurophenomenology strongly emphasizes the micro-dynamics of experience, at the level of brief mental events with very specific content, most neural measures have much coarser functional selectivity, because they mix functionally heterogeneous neural processes either in space or in time. We propose a new starting point for this neurophenomenology, based on (a) the recent development of human intra-cerebral EEG (iEEG) research to highlight the neural micro-dynamics of human cognition, with millimetric and millisecond precision and (b) a disciplined access to the experiential micro-dynamics, through specific elicitation techniques. This lays the foundation for a microcognitive science, the practical implementation of neurophenomenology to combine the neural and experiential investigations of human cognition at the subsecond level.

Global workspace dynamics: cortical "binding and propagation" enables conscious contents

A global workspace (GW) is a functional hub of binding and propagation in a population of loosely coupled signaling elements. In computational applications, GW architectures recruit many distributed, specialized agents to cooperate in resolving focal ambiguities. In the brain, conscious experiences may reflect a GW function. For animals, the natural world is full of unpredictable dangers and opportunities, suggesting a general adaptive pressure for brains to resolve focal ambiguities quickly and accurately. GW theory aims to understand the differences between conscious and unconscious brain events. In humans and related species the cortico-thalamic (C-T) core is believed to underlie conscious aspects of perception, thinking, learning, feelings of knowing (FOK), felt emotions, visual imagery, working memory, and executive control. Alternative theoretical perspectives are also discussed. The C-T core has many anatomical hubs, but conscious percepts are unitary and internally consistent at any given moment. Over time, conscious contents constitute a very large, open set. This suggests that a brain-based GW capacity cannot be localized in a single anatomical hub. Rather, it should be sought in a functional hub - a dynamic capacity for binding and propagation of neural signals over multiple task-related networks, a kind of neuronal cloud computing. In this view, conscious contents can arise in any region of the C-T core when multiple input streams settle on a winner-take-all equilibrium. The resulting conscious gestalt may ignite an any-to-many broadcast, lasting ∼100-200 ms, and trigger widespread adaptation in previously established networks. To account for the great range of conscious contents over time, the theory suggests an open repertoire of binding coalitions that can broadcast via theta/gamma or alpha/gamma phase coupling, like radio channels competing for a narrow frequency band. Conscious moments are thought to hold only 1-4 unrelated items; this small focal capacity may be the biological price to pay for global access. Visuotopic maps in cortex specialize in features like color, retinal size, motion, object identity, and egocentric/allocentric framing, so that a binding coalition for the sight of a rolling billiard ball in nearby space may resonate among activity maps of LGN, V1-V4, MT, IT, as well as the dorsal stream. Spatiotopic activity maps can bind into coherent gestalts using adaptive resonance (reentry). Single neurons can join a dominant coalition by phase tuning to regional oscillations in the 4-12 Hz range. Sensory percepts may bind and broadcast from posterior cortex, while non-sensory FOKs may involve prefrontal and frontotemporal areas. The anatomy and physiology of the hippocampal complex suggest a GW architecture as well. In the intact brain the hippocampal complex may support conscious event organization as well as episodic memory storage.