Neurophysiological bases of exponential sensory decay and top-down memory retrieval: a model

An integrative, multiscale view on neural theories of consciousness

How is conscious experience related to material brain processes? A variety of theories aiming to answer this age-old question have emerged from the recent surge in consciousness research, and some are now hotly debated. Although most researchers have so far focused on the development and validation of their preferred theory in relative isolation, this article, written by a group of scientists representing different theories, takes an alternative approach. Noting that various theories often try to explain different aspects or mechanistic levels of consciousness, we argue that the theories do not necessarily contradict each other. Instead, several of them may converge on fundamental neuronal mechanisms and be partly compatible and complementary, so that multiple theories can simultaneously contribute to our understanding. Here, we consider unifying, integration-oriented approaches that have so far been largely neglected, seeking to combine valuable elements from various theories.

A dynamic neural resource model bridges sensory and working memory

Probing memory of a complex visual image within a few hundred milliseconds after its disappearance reveals significantly greater fidelity of recall than if the probe is delayed by as little as a second. Classically interpreted, the former taps into a detailed but rapidly decaying visual sensory or 'iconic' memory (IM), while the latter relies on capacity-limited but comparatively stable visual working memory (VWM). While iconic decay and VWM capacity have been extensively studied independently, currently no single framework quantitatively accounts for the dynamics of memory fidelity over these time scales. Here, we extend a stationary neural population model of VWM with a temporal dimension, incorporating rapid sensory-driven accumulation of activity encoding each visual feature in memory, and a slower accumulation of internal error that causes memorized features to randomly drift over time. Instead of facilitating read-out from an independent sensory store, an early cue benefits recall by lifting the effective limit on VWM signal strength imposed when multiple items compete for representation, allowing memory for the cued item to be supplemented with information from the decaying sensory trace. Empirical measurements of human recall dynamics validate these predictions while excluding alternative model architectures. A key conclusion is that differences in capacity classically thought to distinguish IM and VWM are in fact contingent upon a single resource-limited WM store.

Hierarchical organization of human physical activity

Human physical activity (HPA), a fundamental physiological signal characteristic of bodily motion is of rapidly growing interest in multidisciplinary research. Here we report the existence of hitherto unidentified hierarchical levels in the temporal organization of HPA on the ultradian scale: on the minute's scale, passive periods are followed by activity bursts of similar intensity ('quanta') that are organized into superstructures on the hours- and on the daily scale. The time course of HPA can be considered a stochastic, quasi-binary process, where quanta, assigned to task-oriented actions are organized into work packages on higher levels of hierarchy. In order to grasp the essence of this complex dynamic behaviour, we established a stochastic mathematical model which could reproduce the main statistical features of real activity time series. The results are expected to provide important data for developing novel behavioural models and advancing the diagnostics of neurological or psychiatric diseases.

Will the real resource theory please stand up! Vigilance is a renewable resource and should be modeled as such

The vigilance decrement or decline in signal detection performance with time on task is one of the most reliable findings in the cognitive neuroscience and psychology literatures. The majority of theories proposed to explain the decrement are limited cognitive or attention resource based theories; the central nervous system is a limited capacity processor. The decrement in performance is then due to resource reallocation (or misallocation), resource depletion or some combination of both mechanisms. The role of resource depletion, in particular, is hotly debated. However, this may be due to a lack of understanding of the renewable nature of the vigilance resources and how this renewal process impacts performance during vigilance tasks. In the present paper, a simple quantitative model of vigilance resource depletion and renewal is described and shown to generate performance data similar to results seen in both humans and spiders. This model clarifies the role resource depletion and resource renewal may play in vigilance in both people and other animals.

A neuronal basis of iconic memory in macaque primary visual cortex

After a briefly presented visual stimulus disappears, observers retain a detailed representation of this stimulus for a short period of time. This sensory storage is called iconic memory. We measured iconic memory in the perception of monkeys and its neuronal correlates in the primary visual cortex (area V1). We determined how many milliseconds extra viewing time iconic memory is worth and how it decays by varying the duration of a brief stimulus and the timing of a mask. The V1 activity that persists after the disappearance of a stimulus predicted accuracy, with a time course resembling the worth and decay of iconic memory. Finally, we examined how iconic memory interacts with attention. A cue presented after the stimulus disappears boosts attentional influences pertaining to a relevant part of the stimulus but only if it appears before iconic memory decayed. Our results relate iconic memory to neuronal activity in early visual cortex.

Stimulus-choice (mis)alignment in primate area MT

For stimuli near perceptual threshold, the trial-by-trial activity of single neurons in many sensory areas is correlated with the animal's perceptual report. This phenomenon has often been attributed to feedforward readout of the neural activity by the downstream decision-making circuits. The interpretation of choice-correlated activity is quite ambiguous, but its meaning can be better understood in the light of population-wide correlations among sensory neurons. Using a statistical nonlinear dimensionality reduction technique on single-trial ensemble recordings from the middle temporal (MT) area during perceptual-decision-making, we extracted low-dimensional latent factors that captured the population-wide fluctuations. We dissected the particular contributions of sensory-driven versus choice-correlated activity in the low-dimensional population code. We found that the latent factors strongly encoded the direction of the stimulus in single dimension with a temporal signature similar to that of single MT neurons. If the downstream circuit were optimally utilizing this information, choice-correlated signals should be aligned with this stimulus encoding dimension. Surprisingly, we found that a large component of the choice information resides in the subspace orthogonal to the stimulus representation inconsistent with the optimal readout view. This misaligned choice information allows the feedforward sensory information to coexist with the decision-making process. The time course of these signals suggest that this misaligned contribution likely is feedback from the downstream areas. We hypothesize that this non-corrupting choice-correlated feedback might be related to learning or reinforcing sensory-motor relations in the sensory population.

Mathematical transcription of the 'time-based resource sharing' theory of working memory

The time-based resource sharing (TBRS) model is a prominent model of working memory that is both predictive and simple. TBRS is a mainstream decay-based model and the most susceptible to competition with interference-based models. A connectionist implementation of TBRS, TBRS*, has recently been developed. However, TBRS* is an enriched version of TBRS, making it difficult to test general characteristics resulting from TBRS assumptions. Here, we describe a novel model, TBRS2, built to be more transparent and simple than TBRS*. TBRS2 is minimalist and allows only a few parameters. It is a straightforward mathematical transcription of TBRS that focuses exclusively on the activation level of memory items as a function of time. Its simplicity makes it possible to derive several theorems from the original TBRS and allows several variants of the refresh process to be tested without relying on particular architectures.

Discrete and continuous mechanisms of temporal selection in rapid visual streams

Humans can reliably detect a target picture even when tens of images are flashed every second. Here we use magnetoencephalography to dissect the neural mechanisms underlying the dynamics of temporal selection during a rapid serial visual presentation task. Multivariate decoding algorithms allow us to track the overlapping brain responses induced by each image in a rapid visual stream. The results show that temporal selection involves a sequence of gradual followed by all-or-none stages: (i) all images first undergo the same parallel processing pipeline; (ii) starting around 150 ms, responses to multiple images surrounding the target are continuously amplified in ventral visual areas; (iii) only the images that are subsequently reported elicit late all-or-none activations in visual and parietal areas around 350 ms. Thus, multiple images can cohabit in the brain and undergo efficient parallel processing, but temporal selection also isolates a single one for amplification and report.

NMDA receptor modulation by dextromethorphan and acute stress selectively alters electroencephalographic indicators of partial report processing

Proceeding from a biophysical network model, the present study hypothesized that glutamatergic neurotransmission across the NMDA receptor (NMDAR) plays a key role in visual perception and its modulation by acute stress. To investigate these hypotheses, behavioral and electroencephalographic (EEG) indicators of partial report task processing were assessed in twenty-four healthy young men who randomly received a non-competitive NMDAR antagonist (0.8 mg/kg dextromethorphan, DXM) or a placebo, and concurrently accomplished a stress-induction (MAST) or control protocol in three consecutive sessions. Saliva samples served to quantify cortisol responses to the MAST, whereas a passive auditory oddball paradigm was implemented to verify the impact of DXM on the EEG-derived mismatch negativity component (MMN). DXM administration significantly increased MMN amplitudes but not salivary cortisol concentrations. By contrast, concurrent MAST exposure significantly reduced MMN latencies but also increased cortisol concentrations. With regard to EEG indicators, DXM administration reduced visually "evoked" (30Hz to 50Hz) and "induced" occipital gamma-band activity (70Hz to 100Hz), which was partly compensated by additional MAST exposure. However, neither the interventions nor EEG activity were significantly associated with behavioral partial report sensitivities. In summary, the present data suggest that glutamatergic neurotransmission across the NMDAR is only one among many determinants of intact visual perception. Accordingly, therapeutic doses of DXM and their inhibitory modulation by stress probably yield more pronounced electroencephalographic as compared with behavioural effects.

Hydrocortisone Counteracts Adverse Stress Effects on Dual-Task Performance by Improving Visual Sensory Processes

The impact of acute stress on executive processes is commonly attributed to glucocorticoid-induced disruptions of the pFC. However, the occipital cortex seems to express a higher density of glucocorticoid receptors. Consequently, acute stress effects on executive processes could as well be mediated by glucocorticoid (e.g., cortisol)-induced alterations of visual sensory processes. To investigate this alternative route of stress action by demarcating the effects of acute stress and cortisol on executive from those on visual sensory processes, 40 healthy young men completed a standardized stress induction (i.e., the Trier Social Stress Test) and control protocol in two consecutive sessions. In addition, they received either a placebo or hydrocortisone (0.12-mg/kg bodyweight) pill and processed a dual and a partial report task to assess their executive and visual sensory processing abilities, respectively. Hydrocortisone administration improved both partial report and dual-task performance as indicated by increased response accuracies and/or decreased RTs. Intriguingly, the hydrocortisone-induced increase in dual-task performance was completely mediated by its impact on partial report performance (i.e., visual sensory processes). Moreover, RT measures in both tasks shared approximately 26% of variance, which was only in part attributable to hydrocortisone administration (ΔR² = 8%). By contrast, acute stress selectively impaired dual-task performance (i.e., executive processes), presumably through an alternative route of action. In summary, the present results suggest that cortisol secretion (as mimicked by hydrocortisone administration) may counteract adverse residual stress effects on executive processes by improving visual sensory processes (e.g., the maintenance and amplification of task-relevant sensory information).

Neural Correlates of Perceived Confidence in a Partial Report Paradigm

Confidence judgments are often severely distorted: People may feel underconfident when responding correctly or, conversely, overconfident in erred responses. Our aim here was to identify the timing of brain processes that lead to variations in objective performance and subjective judgments of confidence. We capitalized on the Partial Report Paradigm [Sperling, G. The information available in brief visual presentations. Psychological Monographs: General and Applied, 74, 1, 1960], which allowed us to separate experimentally the moment of encoding of information from that of its retrieval [Zylberberg, A., Dehaene, S., Mindlin, G. B., & Sigman, M. Neurophysiological bases of exponential sensory decay and top–down memory retrieval: A model. Frontiers in Computational Neuroscience, 3, 2009]. We observed that the level of subjective confidence is indexed by two very specific evoked potentials at latencies of about 400 and 600 msec during the retrieval stage and by a stationary measure of intensity of the alpha band during the encoding period. When factoring out the effect of confidence, objective performance shows a weak effect during the encoding and retrieval periods. These results have relevant implications for theories of decision-making and confidence, suggesting that confidence is not constructed online as evidence is accumulated toward a decision. Instead, confidence attributions are more consistent with a retrospective mechanism that monitors the entire decision process.

Hydrocortisone accelerates the decay of iconic memory traces: on the modulation of executive and stimulus-driven constituents of sensory information maintenance

A substantial amount of research documents the impact of glucocorticoids on higher-order cognitive functioning. By contrast, surprisingly little is known about the susceptibility of basic sensory processes to glucocorticoid exposure given that the glucocorticoid receptor density in the human visual cortex exceeds those observed in prefrontal and most hippocampal brain regions. As executive tasks also rely on these sensory processes, the present study investigates the impact of glucocorticoid exposure on different performance parameters characterizing the maintenance and transfer of sensory information from iconic memory (IM; the sensory buffer of the visual system) to working memory (WM). Using a crossover factorial design, we administered one out of three doses of hydrocortisone (0.06, 0.12, or 0.24mg/kg bodyweight) and a placebo to 18 healthy young men. Thereafter participants performed a partial report task, which was used to assess their individual ability to process sensory information. Blood samples were concurrently drawn to determine free and total cortisol concentrations. The compiled pharmacokinetic and psychophysical data demonstrates that free cortisol specifically accelerated the decay of sensory information (r=0.46) without significantly affecting the selective information transfer from IM to WM or the capacity limit of WM. Specifically, nonparametric regression revealed a sigmoid dose-response relationship between free cortisol levels during the testing period and the IM decay rates. Our findings highlight that glucocorticoid exposure may not only impact on the recruitment of top-down control for an active maintenance of sensory information, but alter their passive (stimulus-driven) maintenance thereby changing the availability of information prior to subsequent executive processing.

Relation of frontal N100 to psychopathy-related differences in selective attention

Research indicates that psychopathy may be characterized by early attentional abnormalities that undermine the processing of peripheral information during goal-directed activity (Baskin-Sommers & Newman, 2012). Past work has found that psychopathic individuals show reduced interference on the Box Stroop task, in which color names are spatially separated from (i.e., peripheral to) colored stimuli (Hiatt, Schmitt, & Newman, 2004). The present study sought to replicate and extend these findings. A priori predictions were that psychopathy scores would be inversely related to interference and that psychopathy-related differences in Box Stroop conflict processing would emerge at an early stage as measured by event-related potentials (ERP). Results supported both hypotheses. Moreover, the association between the early attention-related component (N100) and interference was moderated by level of psychopathy. These findings suggest that psychopathic individuals have less coordinated responses to conflict than healthy individuals, a conjecture that has implications for information integration and self-regulation.

It's hard to forget: resetting memory in delay-match-to-multiple-image tasks

The Delay-Match-to-Sample (DMS) task has been used in countless studies of memory, undergoing numerous modifications, making the task more and more challenging to participants. The physiological correlate of memory is modified neural activity during the cue-to-match delay period reflecting reverberating attractor activity in multiple interconnected cells. DMS tasks may use a fixed set of well-practiced stimulus images-allowing for creation of attractors-or unlimited novel images, for which no attractor exists. Using well-learned stimuli requires that participants determine if a remembered image was seen in the same or a preceding trial, only responding to the former. Thus, trial-to-trial transitions must include a "reset" mechanism to mark old images as such. We test two groups of monkeys on a delay-match-to-multiple-images task, one with well-trained and one with novel images. Only the first developed a reset mechanism. We then switched tasks between the groups. We find that introducing fixed images initiates development of reset, and once established, switching to novel images does not disable its use. Without reset, memory decays slowly, leaving ~40% recognizable after a minute. Here, presence of reward further enhances memory of previously-seen images.

Top-down influences of written text on perceived clarity of degraded speech

An unresolved question is how the reported clarity of degraded speech is enhanced when listeners have prior knowledge of speech content. One account of this phenomenon proposes top-down modulation of early acoustic processing by higher-level linguistic knowledge. Alternative, strictly bottom-up accounts argue that acoustic information and higher-level knowledge are combined at a late decision stage without modulating early acoustic processing. Here we tested top-down and bottom-up accounts using written text to manipulate listeners’ knowledge of speech content. The effect of written text on the reported clarity of noise-vocoded speech was most pronounced when text was presented before (rather than after) speech (Experiment 1). Fine-grained manipulation of the onset asynchrony between text and speech revealed that this effect declined when text was presented more than 120 ms after speech onset (Experiment 2). Finally, the influence of written text was found to arise from phonological (rather than lexical) correspondence between text and speech (Experiment 3). These results suggest that prior knowledge effects are time-limited by the duration of auditory echoic memory for degraded speech, consistent with top-down modulation of early acoustic processing by linguistic knowledge.

Identifying the computational requirements of an integrated top-down-bottom-up model for overt visual attention within an active vision system

Computational visual attention systems have been constructed in order for robots and other devices to detect and locate regions of interest in their visual world. Such systems often attempt to take account of what is known of the human visual system and employ concepts, such as 'active vision', to gain various perceived advantages. However, despite the potential for gaining insights from such experiments, the computational requirements for visual attention processing are often not clearly presented from a biological perspective. This was the primary objective of this study, attained through two specific phases of investigation: 1) conceptual modeling of a top-down-bottom-up framework through critical analysis of the psychophysical and neurophysiological literature, 2) implementation and validation of the model into robotic hardware (as a representative of an active vision system). Seven computational requirements were identified: 1) transformation of retinotopic to egocentric mappings, 2) spatial memory for the purposes of medium-term inhibition of return, 3) synchronization of 'where' and 'what' information from the two visual streams, 4) convergence of top-down and bottom-up information to a centralized point of information processing, 5) a threshold function to elicit saccade action, 6) a function to represent task relevance as a ratio of excitation and inhibition, and 7) derivation of excitation and inhibition values from object-associated feature classes. The model provides further insight into the nature of data representation and transfer between brain regions associated with the vertebrate 'active' visual attention system. In particular, the model lends strong support to the functional role of the lateral intraparietal region of the brain as a primary area of information consolidation that directs putative action through the use of a 'priority map'.

When order matters: last-come first-served effect in sequential arithmetic operations

Cognitive psychologists have relied on dual-task interference experiments to understand the low-capacity and serial nature of conscious mental operations. Two widely studied paradigms, the Attentional Blink (AB) and the Psychological Refractory Period (PRP) have demonstrated a first-come first-served policy; processing a stimulus either impedes conscious access (AB) or postpones treatment (PRP) of a concurrent stimulus. Here we explored the transition from dual-task paradigms to multi-step human cognition. We studied the relative weight of individual addends in a sequential arithmetic task, where number notation (symbolic/non-symbolic) and presentation speed were independently manipulated. For slow presentation and symbolic notation, the decision relied almost equally on all addends, whereas for fast or non-symbolic notation, the decision relied almost exclusively on the last item reflecting a last-come first-served policy. We suggest that streams of stimuli may be chunked in events in which the last stimuli may override previous items from sensory buffers.

Pupil dilation: a fingerprint of temporal selection during the "attentional blink"

Pupil dilation indexes cognitive events of behavioral relevance, like the storage of information to memory and the deployment of attention. Yet, given the slow temporal response of the pupil dilation, it is not known from previous studies whether the pupil can index cognitive events in the short time scale of ∼100 ms. Here we measured the size of the pupil in the Attentional Blink (AB) experiment, a classic demonstration of attentional limitations in processing rapidly presented stimuli. In the AB, two targets embedded in a sequence have to be reported and the second stimulus is often missed if presented between 200 and 500 ms after the first. We show that pupil dilation can be used as a marker of cognitive processing in AB, revealing both the timing and amount of cognitive processing. Specifically, we found that in the time range where the AB is known to occur: (i) the pupil dilation was delayed, mimicking the pattern of response times in the Psychological Refractory Period (PRP) paradigm, (ii) the amplitude of the pupil was reduced relative to that of larger lags, even for correctly identified targets, and (iii) the amplitude of the pupil was smaller for missed than for correctly reported targets. These results support two-stage theories of the Attentional Blink where a second processing stage is delayed inside the interference regime, and indicate that the pupil dilation can be used as a marker of cognitive processing in the time scale of ∼100 ms. Furthermore, given the known relation between the pupil dilation and the activity of the locus coeruleus, our results also support theories that link the serial stage to the action of a specific neuromodulator, norepinephrine.

A temporal bottleneck in the language comprehension network

Humans can understand spoken or written sentences presented at extremely fast rates of ∼400 wpm, far exceeding the normal speech rate (∼150 wpm). How does the brain cope with speeded language? And what processing bottlenecks eventually make language incomprehensible above a certain presentation rate? We used time-resolved fMRI to probe the brain responses to spoken and written sentences presented at five compression rates, ranging from intelligible (60-100% of the natural duration) to challenging (40%) and unintelligible (20%). The results show that cortical areas differ sharply in their activation speed and amplitude. In modality-specific sensory areas, activation varies linearly with stimulus duration. However, a large modality-independent left-hemispheric language network, including the inferior frontal gyrus (pars orbitalis and triangularis) and the superior temporal sulcus, shows a remarkably time-invariant response, followed by a sudden collapse for unintelligible stimuli. Finally, linear and nonlinear responses, reflecting a greater effort as compression increases, are seen at various prefrontal and parietal sites. We show that these profiles fit with a simple model according to which the higher stages of language processing operate at a fixed speed and thus impose a temporal bottleneck on sentence comprehension. At presentation rates faster than this internal processing speed, incoming words must be buffered, and intelligibility vanishes when buffer storage and retrieval operations are saturated. Based on their temporal and amplitude profiles, buffer regions can be identified with the left inferior frontal/anterior insula, precentral cortex, and mesial frontal cortex.

The human Turing machine: a neural framework for mental programs

In recent years much has been learned about how a single computational processing step is implemented in the brain. By contrast, we still have surprisingly little knowledge of the neuronal mechanisms by which multiple such operations are sequentially assembled into mental algorithms. We outline a theory of how individual neural processing steps might be combined into serial programs. We propose a hybrid neuronal device: each step involves massively parallel computation that feeds a slow and serial production system. Production selection is mediated by a system of competing accumulator neurons that extends the role of these neurons beyond the selection of a motor action. Productions change the state of sensory and mnemonic neurons and iteration of such cycles provides a basis for mental programs.

Probing the lifetimes of auditory novelty detection processes

Auditory novelty detection can be fractionated into multiple cognitive processes associated with their respective neurophysiological signatures. In the present study we used high-density scalp event-related potentials (ERPs) during an active version of the auditory oddball paradigm to explore the lifetimes of these processes by varying the stimulus onset asynchrony (SOA). We observed that early MMN (90-160 ms) decreased when the SOA increased, confirming the evanescence of this echoic memory system. Subsequent neural events including late MMN (160-220 ms) and P3a/P3b components of the P3 complex (240-500 ms) did not decay with SOA, but showed a systematic delay effect supporting a two-stage model of accumulation of evidence. On the basis of these observations, we propose a distinction within the MMN complex of two distinct events: (1) an early, pre-attentive and fast-decaying MMN associated with generators located within superior temporal gyri (STG) and frontal cortex, and (2) a late MMN more resistant to SOA, corresponding to the activation of a distributed cortical network including fronto-parietal regions.

The brain's router: a cortical network model of serial processing in the primate brain

The human brain efficiently solves certain operations such as object recognition and categorization through a massively parallel network of dedicated processors. However, human cognition also relies on the ability to perform an arbitrarily large set of tasks by flexibly recombining different processors into a novel chain. This flexibility comes at the cost of a severe slowing down and a seriality of operations (100-500 ms per step). A limit on parallel processing is demonstrated in experimental setups such as the psychological refractory period (PRP) and the attentional blink (AB) in which the processing of an element either significantly delays (PRP) or impedes conscious access (AB) of a second, rapidly presented element. Here we present a spiking-neuron implementation of a cognitive architecture where a large number of local parallel processors assemble together to produce goal-driven behavior. The precise mapping of incoming sensory stimuli onto motor representations relies on a "router" network capable of flexibly interconnecting processors and rapidly changing its configuration from one task to another. Simulations show that, when presented with dual-task stimuli, the network exhibits parallel processing at peripheral sensory levels, a memory buffer capable of keeping the result of sensory processing on hold, and a slow serial performance at the router stage, resulting in a performance bottleneck. The network captures the detailed dynamics of human behavior during dual-task-performance, including both mean RTs and RT distributions, and establishes concrete predictions on neuronal dynamics during dual-task experiments in humans and non-human primates.