Brain oscillations track the formation of episodic memories in the real world

Towards an ecologically valid naturalistic cognitive neuroscience of memory and event cognition

The landscape of human memory and event cognition research has witnessed a transformative journey toward the use of naturalistic contexts and tasks. In this review, we track this progression from abrupt, artificial stimuli used in extensively controlled laboratory experiments to more naturalistic tasks and stimuli that present a more faithful representation of the real world. We argue that in order to improve ecological validity, naturalistic study designs must consider the complexity of the cognitive phenomenon being studied. Then, we review the current state of "naturalistic" event segmentation studies and critically assess frequently employed movie stimuli. We evaluate recently developed tools like lifelogging and other extended reality technologies to help address the challenges we identified with existing naturalistic approaches. We conclude by offering some guidelines that can be used to design ecologically valid cognitive neuroscience studies of memory and event cognition.

Decoding EEG for optimizing naturalistic memory

BACKGROUND

Spectral features of human electroencephalographic (EEG) recordings during learning predict subsequent recall variability.

NEW METHOD

Capitalizing on these fluctuating neural features, we develop a non-invasive closed-loop (NICL) system for real-time optimization of human learning. Participants play a virtual navigation-and-memory game; recording multi-session data across days allowed us to build participant-specific classification models of recall success. In subsequent closed-loop sessions, our platform manipulated the timing of memory encoding, selectively presenting items during periods of predicted good or poor memory function based on EEG features decoded in real time.

RESULTS

The induced memory effect (the difference between recall rates when presenting items during predicted good vs. poor learning periods) increased with the accuracy of neural decoding.

COMPARISON WITH EXISTING METHODS

This study demonstrates greater-than-chance memory decoding from EEG recordings in a naturalistic virtual navigation task with greater real-world validity than basic word-list recall paradigms. Here we modulate memory by timing stimulus presentation based on noninvasive scalp EEG recordings, whereas prior closed-loop studies for memory improvement involved intracranial recordings and direct electrical stimulation. Other noninvasive studies have investigated the use of neurofeedback or remedial study for memory improvement.

CONCLUSIONS

These findings present a proof-of-concept for using non-invasive closed-loop technology to optimize human learning and memory through principled stimulus timing, but only in those participants for whom classifiers reliably predict out-of-sample memory function.

A third kind of episodic memory: Context familiarity is a distinct process from item familiarity and recollection

Episodic memory is accounted for with two processes: 'familiarity' when generally recognizing an item and 'recollection' when retrieving the full contextual details bound with the item. Paradoxically, people sometimes report contextual information as familiar but without recollecting details, which is not easily accounted for by existing theories. We tested a combination of item recognition confidence and source memory, focusing upon 'item-only hits with source unknown' ('item familiarity'), 'low-confidence hits with correct source memory' ('context familiarity'), and 'high-confidence hits with correct source memory' ('recollection'). Results across multiple within-subjects (trial-wise) and between subjects (individual variability) levels indicated these were behaviorally and physiologically distinct. Behaviorally, a crossover interaction was evident in response times, with context familiarity being slower than each condition during item recognition, but faster during source memory. Electrophysiologically, a Condition x Time x Location triple dissociation was evident in event-related potentials (ERPs), which was then independently replicated. Context familiarity exhibited an independent negative central effect from 800-1200 ms, differentiated from positive ERPs for item-familiarity (400 to 600 ms) and recollection (600 to 900 ms). These three conditions thus reflect mutually exclusive, fundamentally different processes of episodic memory. Context familiarity is a third distinct process of episodic memory.

Electrophysiological signatures of veridical head direction in humans

Information about heading direction is critical for navigation as it provides the means to orient ourselves in space. However, given that veridical head-direction signals require physical rotation of the head and most human neuroimaging experiments depend upon fixing the head in position, little is known about how the human brain is tuned to such heading signals. Here we adress this by asking 52 healthy participants undergoing simultaneous electroencephalography and motion tracking recordings (split into two experiments) and 10 patients undergoing simultaneous intracranial electroencephalography and motion tracking recordings to complete a series of orientation tasks in which they made physical head rotations to target positions. We then used a series of forward encoding models and linear mixed-effects models to isolate electrophysiological activity that was specifically tuned to heading direction. We identified a robust posterior central signature that predicts changes in veridical head orientation after regressing out confounds including sensory input and muscular activity. Both source localization and intracranial analysis implicated the medial temporal lobe as the origin of this effect. Subsequent analyses disentangled head-direction signatures from signals relating to head rotation and those reflecting location-specific effects. Lastly, when directly comparing head direction and eye-gaze-related tuning, we found that the brain maintains both codes while actively navigating, with stronger tuning to head direction in the medial temporal lobe. Together, these results reveal a taxonomy of population-level head-direction signals within the human brain that is reminiscent of those reported in the single units of rodents.

Theta and alpha oscillations in human hippocampus and medial parietal cortex support the formation of location-based representations

Our ability to navigate in a new environment depends on learning new locations. Mental representations of locations are quickly accessible during navigation and allow us to know where we are regardless of our current viewpoint. Recent functional magnetic resonance imaging (fMRI) research using pattern classification has shown that these location-based representations emerge in the retrosplenial cortex and parahippocampal gyrus, regions theorized to be critically involved in spatial navigation. However, little is currently known about the oscillatory dynamics that support the formation of location-based representations. We used magnetoencephalogram (MEG) recordings to investigate region-specific oscillatory activity in a task where participants could form location-based representations. Participants viewed videos showing that two perceptually distinct scenes (180° apart) belonged to the same location. This "overlap" video allowed participants to bind the two distinct scenes together into a more coherent location-based representation. Participants also viewed control "non-overlap" videos where two distinct scenes from two different locations were shown, where no location-based representation could be formed. In a post-video behavioral task, participants successfully matched the two viewpoints shown in the overlap videos, but not the non-overlap videos, indicating they successfully learned the locations in the overlap condition. Comparing oscillatory activity between the overlap and non-overlap videos, we found greater theta and alpha/beta power during the overlap relative to non-overlap videos, specifically at time-points when we expected scene integration to occur. These oscillations localized to regions in the medial parietal cortex (precuneus and retrosplenial cortex) and the medial temporal lobe, including the hippocampus. Therefore, we find that theta and alpha/beta oscillations in the hippocampus and medial parietal cortex are likely involved in the formation of location-based representations.

The relationship between childhood interpersonal and non-interpersonal trauma and autobiographical memory: a systematic review

Childhood trauma can have negative effects on several domains of mental functioning, including Autobiographical Memory (AM). Conflicting results emerge in the scientific literature regarding the effects of childhood trauma on AM. In this review, we explored the relationship between the childhood trauma and AM, classifying childhood trauma as interpersonal, non-interpersonal and overall (interpersonal and non-interpersonal). We carried out a systematic literature review, following the guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA statement). From searching the PubMed, Scopus, and Web of Science databases, we identified 48 studies conducted from 2014 to 2023, which were included when they: (a) were written in English, (b) investigated the relationship between AM and childhood trauma, (c) included a sample of children, adolescents, or adults who had experienced childhood interpersonal and/or non-interpersonal trauma. Of the 48 eligible studies, 29 referred to trauma of an interpersonal nature, 12 to trauma of a non-interpersonal nature, and 7 to overall trauma. Regarding the relationship between childhood trauma and AM, 24 studies found a negative relationship between childhood interpersonal trauma and AM; among the articles on non-interpersonal trauma, 10 studies found no relevant relationship; in the studies on overall trauma, 4 articles found negative relationship between overall trauma and AM. The literature explored in our systematic review supports the prevalence of a negative relationship between interpersonal childhood trauma and AM. This relationship is present regardless of psychiatric disorders (e.g., Depression, Post Traumatic Stress Disorder, and Personality Disorders), and in the presence of the latter, AM results even more fragmented. Future research should use more accurate methodologies in identifying and classifying childhood trauma in order to more precisely determine its effect on AM.

Temporal dynamics of oscillatory activity during nonlexical language decoding: Evidence from Morse code and magnetoencephalography

Understanding encoded languages, such as written script or Morse code, requires nonlexical and lexical processing components that act in a parallel and interactive fashion. Decoding written script-as for example in reading-is typically very fast, making the investigation of the lexical and nonlexical components and their underlying neural mechanisms challenging. In the current study, we aimed to accomplish this problem by using Morse code as a model for language decoding. The decoding of Morse code is slower and thus allows a better and more fine-grained investigation of the lexical and nonlexical components of language decoding. In the current study, we investigated the impact of various components of nonlexical decoding of Morse code using magnetoencephalography. For this purpose, we reconstructed the time-frequency responses below 40 Hz in brain regions significantly involved in Morse code decoding and word comprehension that were identified in a previous study. Event-related reduction in beta- and alpha-band power were found in left inferior frontal cortex and angular gyrus, respectively, while event-related theta-band power increase was found at frontal midline. These induced oscillations reflect working-memory encoding, long-term memory retrieval as well as demanding cognitive control, respectively. In sum, by using Morse code and MEG, we were able to identify a cortical network underlying language decoding in a time- and frequency-resolved manner.

Theta oscillatory power decreases in humans are associated with spatial learning in a virtual water maze task

Theta oscillations (4-8 Hz) in humans play a role in navigation processes, including spatial encoding, retrieval and sensorimotor integration. Increased theta power at frontal and parietal midline regions is known to contribute to successful navigation. However, the dynamics of cortical theta and its role in spatial learning are not fully understood. This study aimed to investigate theta oscillations via electroencephalogram (EEG) during spatial learning in a virtual water maze. Participants were separated into a learning group (n = 25) who learned the location of a hidden goal across 12 trials, or a time-matched non-learning group (n = 25) who were required to simply navigate the same arena, but without a goal. We compared all trials, at two phases of learning, the trial start and the goal approach. We also compared the first six trials with the last six trials within-groups. The learning group showed reduced low-frequency theta power at the frontal and parietal midline during the start phase and largely reduced theta combined with a short increase at both midlines during the goal-approach phase. These patterns were not found in the non-learning group, who instead displayed extensive increases in low-frequency oscillations at both regions during the trial start and at the parietal midline during goal approach. Our results support the theory that theta plays a crucial role in spatial encoding during exploration, as opposed to sensorimotor integration. We suggest our findings provide evidence for a link between learning and a reduction of theta oscillations in humans.

An ERP measure of non-conscious memory reveals dissociable implicit processes in human recognition using an open-source automated analytic pipeline

Non-conscious processing of human memory has traditionally been difficult to objectively measure and thus understand. A prior study on a group of hippocampal amnesia (N = 3) patients and healthy controls (N = 6) used a novel procedure for capturing neural correlates of implicit memory using event-related potentials (ERPs): old and new items were equated for varying levels of memory awareness, with ERP differences observed from 400 to 800 ms in bilateral parietal regions that were hippocampal-dependent. The current investigation sought to address the limitations of that study by increasing the sample of healthy subjects (N = 54), applying new controls for construct validity, and developing an improved, open-source tool for automated analysis of the procedure used for equating levels of memory awareness. Results faithfully reproduced prior ERP findings of parietal effects that a series of systematic control analyses validated were not contributed to nor contaminated by explicit memory. Implicit memory effects extended from 600 to 1000 ms, localized to right parietal sites. These ERP effects were found to be behaviorally relevant and specific in predicting implicit memory response times, and were topographically dissociable from other traditional ERP measures of implicit memory (miss vs. correct rejections) that instead occurred in left parietal regions. Results suggest first that equating for reported awareness of memory strength is a valid, powerful new method for revealing neural correlates of non-conscious human memory, and second, behavioral correlations suggest that these implicit effects reflect a pure form of priming, whereas misses represent fluency leading to the subjective experience of familiarity.

Gamma oscillations and episodic memory

Enhanced gamma oscillatory activity (30-80 Hz) accompanies the successful formation and retrieval of episodic memories. While this co-occurrence is well documented, the mechanistic contributions of gamma oscillatory activity to episodic memory remain unclear. Here, we review how gamma oscillatory activity may facilitate spike timing-dependent plasticity, neural communication, and sequence encoding/retrieval, thereby ensuring the successful formation and/or retrieval of an episodic memory. Based on the evidence reviewed, we propose that multiple, distinct forms of gamma oscillation can be found within the canonical gamma band, each of which has a complementary role in the neural processes listed above. Further exploration of these theories using causal manipulations may be key to elucidating the relevance of gamma oscillatory activity to episodic memory.

Mobile cognition: imaging the human brain in the 'real world'

Cognitive neuroscience studies in humans have enabled decades of impactful discoveries but have primarily been limited to recording the brain activity of immobile participants in a laboratory setting. In recent years, advances in neuroimaging technologies have enabled recordings of human brain activity to be obtained during freely moving behaviours in the real world. Here, we propose that these mobile neuroimaging methods can provide unique insights into the neural mechanisms of human cognition and contribute to the development of novel treatments for neurological and psychiatric disorders. We further discuss the challenges associated with studying naturalistic human behaviours in complex real-world settings as well as strategies for overcoming them. We conclude that mobile neuroimaging methods have the potential to bring about a new era of cognitive neuroscience in which neural mechanisms can be studied with increased ecological validity and with the ability to address questions about natural behaviour and cognitive processes in humans engaged in dynamic real-world experiences.

Direct brain recordings suggest a causal subsequent-memory effect

Endogenous variation in brain state and stimulus-specific evoked activity can both contribute to successful encoding. Previous studies, however, have not clearly distinguished among these components. We address this question by analysing intracranial EEG recorded from epilepsy patients as they studied and subsequently recalled lists of words. We first trained classifiers to predict recall of either single items or entire lists and found that both classifiers exhibited similar performance. We found that list-level classifier output-a biomarker of successful encoding-tracked item presentation and recall events, despite having no information about the trial structure. Across widespread brain regions, decreased low- and increased high-frequency activity (HFA) marked successful encoding of both items and lists. We found regional differences in the hippocampus and prefrontal cortex, where in the hippocampus HFA correlated more strongly with item recall, whereas, in the prefrontal cortex, HFA correlated more strongly with list performance. Despite subtle differences in item- and list-level features, the similarity in overall classification performance, spectral signatures of successful recall and fluctuations of spectral activity across the encoding period argue for a shared endogenous process that causally impacts the brain's ability to learn new information.

Sleep loss disrupts the neural signature of successful learning

Sleep supports memory consolidation as well as next-day learning. The influential "Active Systems" account of offline consolidation suggests that sleep-associated memory processing paves the way for new learning, but empirical evidence in support of this idea is scarce. Using a within-subjects (n = 30), crossover design, we assessed behavioral and electrophysiological indices of episodic encoding after a night of sleep or total sleep deprivation in healthy adults (aged 18-25 years) and investigated whether behavioral performance was predicted by the overnight consolidation of episodic associations from the previous day. Sleep supported memory consolidation and next-day learning as compared to sleep deprivation. However, the magnitude of this sleep-associated consolidation benefit did not significantly predict the ability to form novel memories after sleep. Interestingly, sleep deprivation prompted a qualitative change in the neural signature of encoding: Whereas 12-20 Hz beta desynchronization-an established marker of successful encoding-was observed after sleep, sleep deprivation disrupted beta desynchrony during successful learning. Taken together, these findings suggest that effective learning depends on sleep but not necessarily on sleep-associated consolidation.

Functional near-infrared spectroscopy is a useful tool for multi-perspective psychobiological study of neurophysiological correlates of parenting behaviour

The quality of the relationship between caregiver and child has long-term effects on the cognitive and socio-emotional development of children. A process involved in human parenting is the bio-behavioural synchrony that occurs between the partners in the relationship during interaction. Through interaction, bio-behavioural synchronicity allows the adaptation of the physiological systems of the parent to those of the child and promotes the positive development and modelling of the child's social brain. The role of bio-behavioural synchrony in building social bonds could be investigated using functional near-infrared spectroscopy (fNIRS). In this paper we have (a) highlighted the importance of the quality of the caregiver-child relationship for the child's cognitive and socio-emotional development, as well as the relevance of infantile stimuli in the activation of parenting behaviour; (b) discussed the tools used in the study of the neurophysiological substrates of the parental response; (c) proposed fNIRS as a particularly suitable tool for the study of parental responses; and (d) underlined the need for a multi-systemic psychobiological approach to understand the mechanisms that regulate caregiver-child interactions and their bio-behavioural synchrony. We propose to adopt a multi-system psychobiological approach to the study of parental behaviour and social interaction.

Rhythms of human attention and memory: An embedded process perspective

It remains a dogma in cognitive neuroscience to separate human attention and memory into distinct modules and processes. Here we propose that brain rhythms reflect the embedded nature of these processes in the human brain, as evident from their shared neural signatures: gamma oscillations (30-90 Hz) reflect sensory information processing and activated neural representations (memory items). The theta rhythm (3-8 Hz) is a pacemaker of explicit control processes (central executive), structuring neural information processing, bit by bit, as reflected in the theta-gamma code. By representing memory items in a sequential and time-compressed manner the theta-gamma code is hypothesized to solve key problems of neural computation: (1) attentional sampling (integrating and segregating information processing), (2) mnemonic updating (implementing Hebbian learning), and (3) predictive coding (advancing information processing ahead of the real time to guide behavior). In this framework, reduced alpha oscillations (8-14 Hz) reflect activated semantic networks, involved in both explicit and implicit mnemonic processes. Linking recent theoretical accounts and empirical insights on neural rhythms to the embedded-process model advances our understanding of the integrated nature of attention and memory - as the bedrock of human cognition.

Mapping and understanding of correlated electroencephalogram (EEG) responses to the newsvendor problem

Decision-making is one of the most critical activities of human beings. To better understand the underlying neurocognitive mechanism while making decisions under an economic context, we designed a decision-making paradigm based on the newsvendor problem (NP) with two scenarios: low-profit margins as the more challenging scenario and high-profit margins as the less difficult one. The EEG signals were acquired from healthy humans while subjects were performing the task. We adopted the Correlated Component Analysis (CorrCA) method to identify linear combinations of EEG channels that maximize the correlation across subjects ([Formula: see text]) or trials ([Formula: see text]). The inter-subject or inter-trial correlation values (ISC or ITC) of the first three components were estimated to investigate the modulation of the task difficulty on subjects' EEG signals and respective correlations. We also calculated the alpha- and beta-band power of the projection components obtained by the CorrCA to assess the brain responses across multiple task periods. Finally, the CorrCA forward models, which represent the scalp projections of the brain activities by the maximally correlated components, were further translated into source distributions of underlying cortical activity using the exact Low Resolution Electromagnetic Tomography Algorithm (eLORETA). Our results revealed strong and significant correlations in EEG signals among multiple subjects and trials during the more difficult decision-making task than the easier one. We also observed that the NP decision-making and feedback tasks desynchronized the normalized alpha and beta powers of the CorrCA components, reflecting the engagement state of subjects. Source localization results furthermore suggested several sources of neural activities during the NP decision-making process, including the dorsolateral prefrontal cortex, anterior PFC, orbitofrontal cortex, posterior cingulate cortex, and somatosensory association cortex.

Sleep spindles track cortical learning patterns for memory consolidation

Memory consolidation-the transformation of labile memory traces into stable long-term representations-is facilitated by post-learning sleep. Computational and biophysical models suggest that sleep spindles may play a key mechanistic role for consolidation, igniting structural changes at cortical sites involved in prior learning. Here, we tested the resulting prediction that spindles are most pronounced over learning-related cortical areas and that the extent of this learning-spindle overlap predicts behavioral measures of memory consolidation. Using high-density scalp electroencephalography (EEG) and polysomnography (PSG) in healthy volunteers, we first identified cortical areas engaged during a temporospatial associative memory task (power decreases in the alpha/beta frequency range, 6-20 Hz). Critically, we found that participant-specific topographies (i.e., spatial distributions) of post-learning sleep spindle amplitude correlated with participant-specific learning topographies. Importantly, the extent to which spindles tracked learning patterns further predicted memory consolidation across participants. Our results provide empirical evidence for a role of post-learning sleep spindles in tracking learning networks, thereby facilitating memory consolidation.

Towards real-world neuroscience using mobile EEG and augmented reality

Our visual environment impacts multiple aspects of cognition including perception, attention and memory, yet most studies traditionally remove or control the external environment. As a result, we have a limited understanding of neurocognitive processes beyond the controlled lab environment. Here, we aim to study neural processes in real-world environments, while also maintaining a degree of control over perception. To achieve this, we combined mobile EEG (mEEG) and augmented reality (AR), which allows us to place virtual objects into the real world. We validated this AR and mEEG approach using a well-characterised cognitive response-the face inversion effect. Participants viewed upright and inverted faces in three EEG tasks (1) a lab-based computer task, (2) walking through an indoor environment while seeing face photographs, and (3) walking through an indoor environment while seeing virtual faces. We find greater low frequency EEG activity for inverted compared to upright faces in all experimental tasks, demonstrating that cognitively relevant signals can be extracted from mEEG and AR paradigms. This was established in both an epoch-based analysis aligned to face events, and a GLM-based approach that incorporates continuous EEG signals and face perception states. Together, this research helps pave the way to exploring neurocognitive processes in real-world environments while maintaining experimental control using AR.

Stimulation of the left dorsolateral prefrontal cortex with slow rTMS enhances verbal memory formation

Encoding of episodic memories relies on stimulus-specific information processing and involves the left prefrontal cortex. We here present an incidental finding from a simultaneous EEG-TMS experiment as well as a replication of this unexpected effect. Our results reveal that stimulating the left dorsolateral prefrontal cortex (DLPFC) with slow repetitive transcranial magnetic stimulation (rTMS) leads to enhanced word memory performance. A total of 40 healthy human participants engaged in a list learning paradigm. Half of the participants (N = 20) received 1 Hz rTMS to the left DLPFC, while the other half (N = 20) received 1 Hz rTMS to the vertex and served as a control group. Participants receiving left DLPFC stimulation demonstrated enhanced memory performance compared to the control group. This effect was replicated in a within-subjects experiment where 24 participants received 1 Hz rTMS to the left DLPFC and vertex. In this second experiment, DLPFC stimulation also induced better memory performance compared to vertex stimulation. In addition to these behavioural effects, we found that 1 Hz rTMS to DLPFC induced stronger beta power modulation in posterior areas, a state that is known to be beneficial for memory encoding. Further analysis indicated that beta modulations did not have an oscillatory origin. Instead, the observed beta modulations were a result of a spectral tilt, suggesting inhibition of these parietal regions. These results show that applying 1 Hz rTMS to DLPFC, an area involved in episodic memory formation, improves memory performance via modulating neural activity in parietal regions.

Encoding of episodic memories relies on stimulus-specific information processing and involves the left prefrontal cortex. An incidental finding from a simultaneous EEG-TMS experiment reveals that applying 1-Hz repetitive transcranial magnetic stimulation to this area of the brain improves memory performance by modulating neural activity in parietal regions.

The phase of Theta oscillations modulates successful memory formation at encoding

Several studies have shown that attention and perception can depend upon the phase of ongoing neural oscillations at stimulus onset. Here, we extend this idea to the memory domain. We tested the hypothesis that ongoing fluctuations in neural activity impact memory encoding in two experiments using a picture paired-associates task in order to gauge episodic memory performance. Experiment 1 was behavioural only and capitalized on the principle of phase resetting. We tested if subsequent memory performance fluctuates rhythmically, time-locked to a resetting cue presented before the to-be-remembered pairs at different time intervals. We found an indication that behavioural performance was periodically and selectively modulated at Theta frequency (~4 Hz). In Experiment 2, we focused on pre-stimulus ongoing activity using scalp EEG while participants performed a paired-associates task. The pre-registered analysis, using large electrode clusters and generic Theta and Alpha spectral ranges, returned null results of the pre-stimulus phase-behaviour correlation. However, as expected from prior literature, we found that variations in stimulus-related Theta-power predicted subsequent memory performance. Therefore, we used this post-stimulus effect in Theta power to guide a post-hoc pre-stimulus phase analysis in terms of scalp and frequency of interest. This analysis returned a correlation between the pre-stimulus Theta phase and subsequent memory. Altogether, these results suggest that pre-stimulus Theta activity at encoding may impact later memory performance.

Alpha/beta power decreases during episodic memory formation predict the magnitude of alpha/beta power decreases during subsequent retrieval

Episodic memory retrieval is characterised by the vivid reinstatement of information about a personally-experienced event. Growing evidence suggests that this reinstatement is supported by reductions in the spectral power of alpha/beta activity. Given that the amount of information that can be recalled depends on the amount of information that was originally encoded, information-based accounts of alpha/beta activity would suggest that retrieval-related alpha/beta power decreases similarly depend upon decreases in alpha/beta power during encoding. To test this hypothesis, seventeen human participants completed a sequence-learning task while undergoing concurrent MEG recordings. Regression-based analyses were then used to estimate how alpha/beta power decreases during encoding predicted alpha/beta power decreases during retrieval on a trial-by-trial basis. When subjecting these parameter estimates to group-level analysis, we find evidence to suggest that retrieval-related alpha/beta (7-15Hz) power decreases fluctuate as a function of encoding-related alpha/beta power decreases. These results suggest that retrieval-related alpha/beta power decreases are contingent on the decrease in alpha/beta power that arose during encoding. Subsequent analysis uncovered no evidence to suggest that these alpha/beta power decreases reflect stimulus identity, indicating that the contingency between encoding- and retrieval-related alpha/beta power reflects the reinstatement of a neurophysiological operation, rather than neural representation, during episodic memory retrieval.

Modulation of attention and stress with arousal: The mental and physical effects of riding a motorcycle

Existing theories suggest that moderate arousal improves selective attention, as would be expected in the context of competitive sports or sensation-seeking activities. Here we investigated how riding a motorcycle, an attention-demanding physical activity, affects sensory processing. To do so, we implemented the passive auditory oddball paradigm and measured the EEG response of participants as they rode a motorcycle, drove a car, and sat at rest. Specifically, we measured the N1 and mismatch negativity to auditory tones, as well as alpha power during periods of no tones. We investigated whether riding and driving modulated non-CNS metrics including heart rate and concentrations of the hormones epinephrine, cortisol, DHEA-S, and testosterone. While participants were riding, we found a decrease in N1 amplitude, increase in mismatch negativity, and decrease in relative alpha power, together suggesting enhancement of sensory processing and visual attention. Riding increased epinephrine levels, increased heart rate, and decreased the ratio of cortisol to DHEA-S. Together, these results suggest that riding increases focus, heightens the brain's passive monitoring of changes in the sensory environment, and alters HPA axis response. More generally, our findings suggest that selective attention and sensory monitoring seem to be separable neural processes.

The Modulation of Cognitive Performance with Transcranial Alternating Current Stimulation: A Systematic Review of Frequency-Specific Effects

Transcranial alternating current stimulation (tACS) is a non-invasive brain stimulation technique that allows the manipulation of intrinsic brain oscillations. Numerous studies have applied tACS in the laboratory to enhance cognitive performance. With this systematic review, we aim to provide an overview of frequency-specific tACS effects on a range of cognitive functions in healthy adults. This may help to transfer stimulation protocols to real-world applications. We conducted a systematic literature search on PubMed and Cochrane databases and considered tACS studies in healthy adults (age > 18 years) that focused on cognitive performance. The search yielded n = 109 studies, of which n = 57 met the inclusion criteria. The results indicate that theta-tACS was beneficial for several cognitive functions, including working memory, executive functions, and declarative memory. Gamma-tACS enhanced performance in both auditory and visual perception but it did not change performance in tasks of executive functions. For attention, the results were less consistent but point to an improvement in performance with alpha- or gamma-tACS. We discuss these findings and point to important considerations that would precede a transfer to real-world applications.

Commonalities in alpha and beta neural desynchronizations during prediction in language comprehension and production

The present study investigates whether predictions during language comprehension are generated by engaging the language production system. Previous studies investigating either prediction or production highlighted M/EEG desynchronization (power decrease) in the alpha (8-10 Hz) and beta (13-30 Hz) frequency bands preceding the target. However, it is unclear whether this electrophysiological modulation underlies common mechanisms. We recorded EEG from participants performing both a comprehension and a production task in two separate blocks. Participants listened to high and low constraint incomplete sentences and were asked either to name a picture to complete them (production) or to simply listen to the final word (comprehension). We found that in a silent gap before the final stimulus, predictable stimuli elicited alpha and beta desynchronization in both tasks, signaling the pre-activation of linguistic information. Source estimation highlighted the involvement of left-lateralized language areas and temporo-parietal areas in the right hemisphere. Furthermore, correlations between the desynchronizations in comprehension and production showed spatiotemporal commonalities in language-relevant areas of the left hemisphere. As proposed by prediction-by-production models, our results suggest that comprehenders engage the production system while predicting upcoming words.

Using Mobile EEG to Investigate Alpha and Beta Asymmetries During Hand and Foot Use

The Edinburgh Handedness Inventory (EHI) and the Waterloo Footedness Questionnaire (WFQ) are two of the most widely used questionnaires to assess lateralized everyday behavior in human participants. However, it is unclear to what extent the specific behavior assessed in these questionnaires elicit lateralized neural activity when performed in real-life situations. To illuminate this unresolved issue, we assessed EEG alpha and beta asymmetries during real-life performance of the behaviors assessed in the EHI and WFQ using a mobile EEG system. This methodology provides high ecological validity for studying neural correlates of motor behavior under more naturalistic conditions. Our results indicate that behavioral performance of items of both the EHI and WFQ differentiate between left- and right-handers and left- and right-footers on the neural level, especially in the alpha frequency band. These results were unaffected by movement parameters. Furthermore, we could demonstrate that neural activity elicited specifically during left-sided task performance provides predictive power for the EHI or WFQ score of the participants. Overall, our results show that these prominent questionnaires not only distinguish between different motor preferences on the behavioral level, but also on the neurophysiological level. Furthermore, we could show that mobile EEG systems are a powerful tool to investigate motor asymmetries in ecologically valid situations outside of the laboratory setting. Future research should focus on other lateralized behavioral phenotypes in real-life settings to provide more insights into lateralized motor functions.

Imaging the human hippocampus with optically-pumped magnetoencephalography

Optically-pumped (OP) magnetometers allow magnetoencephalography (MEG) to be performed while a participant's head is unconstrained. To fully leverage this new technology, and in particular its capacity for mobility, the activity of deep brain structures which facilitate explorative behaviours such as navigation, must be detectable using OP-MEG. One such crucial brain region is the hippocampus. Here we had three healthy adult participants perform a hippocampal-dependent task - the imagination of novel scene imagery - while being scanned using OP-MEG. A conjunction analysis across these three participants revealed a significant change in theta power in the medial temporal lobe. The peak of this activated cluster was located in the anterior hippocampus. We repeated the experiment with the same participants in a conventional SQUID-MEG scanner and found similar engagement of the medial temporal lobe, also with a peak in the anterior hippocampus. These OP-MEG findings indicate exciting new opportunities for investigating the neural correlates of a range of crucial cognitive functions in naturalistic contexts including spatial navigation, episodic memory and social interactions.

Alpha/beta power decreases track the fidelity of stimulus-specific information

Massed synchronised neuronal firing is detrimental to information processing. When networks of task-irrelevant neurons fire in unison, they mask the signal generated by task-critical neurons. On a macroscopic level, such synchronisation can contribute to alpha/beta (8-30 Hz) oscillations. Reducing the amplitude of these oscillations, therefore, may enhance information processing. Here, we test this hypothesis. Twenty-one participants completed an associative memory task while undergoing simultaneous EEG-fMRI recordings. Using representational similarity analysis, we quantified the amount of stimulus-specific information represented within the BOLD signal on every trial. When correlating this metric with concurrently-recorded alpha/beta power, we found a significant negative correlation which indicated that as post-stimulus alpha/beta power decreased, stimulus-specific information increased. Critically, we found this effect in three unique tasks: visual perception, auditory perception, and visual memory retrieval, indicating that this phenomenon transcends both stimulus modality and cognitive task. These results indicate that alpha/beta power decreases parametrically track the fidelity of both externally-presented and internally-generated stimulus-specific information represented within the cortex.

Directional coupling of slow and fast hippocampal gamma with neocortical alpha/beta oscillations in human episodic memory

Episodic memories hinge upon our ability to process a wide range of multisensory information and bind this information into a coherent, memorable representation. On a neural level, these 2 processes are thought to be supported by neocortical alpha/beta desynchronization and hippocampal theta/gamma synchronization, respectively. Intuitively, these 2 processes should couple to successfully create and retrieve episodic memories, yet this hypothesis has not been tested empirically. We address this by analyzing human intracranial electroencephalogram data recorded during 2 associative memory tasks. We find that neocortical alpha/beta (8 to 20 Hz) power decreases reliably precede and predict hippocampal "fast" gamma (60 to 80 Hz) power increases during episodic memory formation; during episodic memory retrieval, however, hippocampal "slow" gamma (40 to 50 Hz) power increases reliably precede and predict later neocortical alpha/beta power decreases. We speculate that this coupling reflects the flow of information from the neocortex to the hippocampus during memory formation, and hippocampal pattern completion inducing information reinstatement in the neocortex during memory retrieval.

Event conjunction: How the hippocampus integrates episodic memories across event boundaries

Our lives are a continuous stream of experience. Our episodic memories on the other hand have a definitive beginning, middle, and end. Theories of event segmentation suggest that salient changes in our environment produce event boundaries which partition the past from the present and, as a result, produce discretized memories. However, event boundaries cannot completely discretize two memories; any shared conceptual link will lead to the rapid integration of these memories. Here, we present a new framework inspired by electrophysiological research that resolves this apparent contradiction. At its heart, the framework proposes that hippocampal theta-gamma coupling maintains a highly abstract model of an ongoing event and serves to encode this model as an episodic memory. When a second but related event begins, this theta-gamma model is rapidly reconstructed within the hippocampus where new details of the second event can be appended to the existing event model. The event conjunction framework is the first electrophysiological explanation of how event memories can be formed at, and integrated across, event boundaries.

Real-Life Neuroscience: An Ecological Approach to Brain and Behavior Research

Owing to advances in neuroimaging technology, the past couple of decades have witnessed a surge of research on brain mechanisms that underlie human cognition. Despite the immense development in cognitive neuroscience, the vast majority of neuroimaging experiments examine isolated agents carrying out artificial tasks in sensory and socially deprived environments. Thus, the understanding of the mechanisms of various domains in cognitive neuroscience, including social cognition and episodic memory, is sorely lacking. Here we focus on social and memory research as representatives of cognitive functions and propose that mainstream, lab-based experimental designs in these fields suffer from two fundamental limitations, pertaining to person-dependent and situation-dependent factors. The person-dependent factor addresses the issue of limiting the active role of the participants in lab-based paradigms that may interfere with their sense of agency and embodiment. The situation-dependent factor addresses the issue of the artificial decontextualized environment in most available paradigms. Building on recent findings showing that real-life as opposed to controlled experimental paradigms involve different mechanisms, we argue that adopting a real-life approach may radically change our understanding of brain and behavior. Therefore, we advocate in favor of a paradigm shift toward a nonreductionist approach, exploiting portable technology in semicontrolled environments, to explore behavior in real life.

Age Differences in Encoding-Related Alpha Power Reflect Sentence Comprehension Difficulties

When sentence processing taxes verbal working memory, comprehension difficulties arise. This is specifically the case when processing resources decline with advancing adult age. Such decline likely affects the encoding of sentences into working memory, which constitutes the basis for successful comprehension. To assess age differences in encoding-related electrophysiological activity, we recorded the electroencephalogram from three age groups (24, 43, and 65 years). Using an auditory sentence comprehension task, age differences in encoding-related oscillatory power were examined with respect to the accuracy of the given response. That is, the difference in oscillatory power between correctly and incorrectly encoded sentences, yielding subsequent memory effects (SME), was compared across age groups. Across age groups, we observed an age-related SME inversion in the alpha band from a power decrease in younger adults to a power increase in older adults. We suggest that this SME inversion underlies age-related comprehension difficulties. With alpha being commonly linked to inhibitory processes, this shift may reflect a change in the cortical inhibition-disinhibition balance. A cortical disinhibition may imply enriched sentence encoding in younger adults. In contrast, resource limitations in older adults may necessitate an increase in cortical inhibition during sentence encoding to avoid an information overload. Overall, our findings tentatively suggest that age-related comprehension difficulties are associated with alterations to the electrophysiological dynamics subserving general higher cognitive functions.

Mobile EEG in research on neurodevelopmental disorders: Opportunities and challenges

Mobile electroencephalography (mobile EEG) represents a next-generation neuroscientific technology – to study real-time brain activity – that is relatively inexpensive, non-invasive and portable. Mobile EEG leverages state-of-the-art hardware alongside established advantages of traditional EEG and recent advances in signal processing. In this review, we propose that mobile EEG could open unprecedented possibilities for studying neurodevelopmental disorders. We first present a brief overview of recent developments in mobile EEG technologies, emphasising the proliferation of studies in several neuroscientific domains. As these developments have yet to be exploited by neurodevelopmentalists, we then identify three research opportunities: 1) increase in the ease and flexibility of brain data acquisition in neurodevelopmental populations; 2) integration into powerful developmentally-informative research designs; 3) development of innovative non-stationary EEG-based paradigms. Critically, we address key challenges that should be considered to fully realise the potential of mobile EEG for neurodevelopmental research and for understanding developmental psychopathology more broadly, and suggest future research directions.

Characterizing hippocampal dynamics with MEG: A systematic review and evidence-based guidelines

The hippocampus, a hub of activity for a variety of important cognitive processes, is a target of increasing interest for researchers and clinicians. Magnetoencephalography (MEG) is an attractive technique for imaging spectro-temporal aspects of function, for example, neural oscillations and network timing, especially in shallow cortical structures. However, the decrease in MEG signal-to-noise ratio as a function of source depth implies that the utility of MEG for investigations of deeper brain structures, including the hippocampus, is less clear. To determine whether MEG can be used to detect and localize activity from the hippocampus, we executed a systematic review of the existing literature and found successful detection of oscillatory neural activity originating in the hippocampus with MEG. Prerequisites are the use of established experimental paradigms, adequate coregistration, forward modeling, analysis methods, optimization of signal-to-noise ratios, and protocol trial designs that maximize contrast for hippocampal activity while minimizing those from other brain regions. While localizing activity to specific sub-structures within the hippocampus has not been achieved, we provide recommendations for improving the reliability of such endeavors.

Out and about: Subsequent memory effect captured in a natural outdoor environment with smartphone EEG

Spatiotemporal context plays an important role in episodic memory. While temporal context effects have been frequently studied in the laboratory, ecologically valid spatial context manipulations are difficult to implement in stationary conditions. We investigated whether the neural correlates of successful encoding (subsequent memory effect) can be captured in a real-world environment. An off-the-shelf Android smartphone was used for wireless mobile EEG acquisition and stimulus presentation. Participants encoded single words, each of which was presented at a different location on a university campus. Locations were approximately 10-12 m away from each other, half of them with striking features (landmarks) nearby. We predicted landmarks would improve recall performance. After a first free recall task of verbal stimuli indoors, participants performed a subsequent recall outdoors, in which words and locations were recalled. As predicted, significantly more words presented at landmark locations as well as significantly more landmark than nonlandmark locations were recalled. ERP analysis yielded a larger posterior positive deflection during encoding for hits compared to misses in the 400-800 ms interval. Likewise, time-frequency analysis revealed a significant difference during encoding for hits compared to misses in the form of stronger alpha (200-300 ms) and theta (300-400 ms) power increases. Our results confirm that a vibrant spatial context is beneficial in episodic memory processing and that the underlying neural correlates can be captured with unobtrusive smartphone EEG technology. The advent of mobile EEG technology promises to unveil the relevance of natural physical activity and natural environments on memory.

Theta-gamma coupling binds visual perceptual features in an associative memory task

It is an integral function of the human brain to sample novel information from the environment and to integrate them into existing representations. Recent evidence suggests a specific role for the theta rhythm (4-8 Hz) in mnemonic processes and the coupling between the theta and the gamma rhythm (40-120 Hz) in ordering and binding perceptual features during encoding. Furthermore, decreases in the alpha rhythm (8-12 Hz) are assumed to gate perceptual information processes in semantic networks. In the present study, we used an associative memory task (object-color combinations) with pictures versus words as stimuli (high versus low visual information) to separate associative memory from visual perceptual processes during memory formation. We found increased theta power for later remembered versus later forgotten items (independent of the color judgement) and an increase in phase-amplitude coupling between frontal theta and fronto-temporal gamma oscillations, specific for the formation of picture-color associations. Furthermore, parietal alpha suppression and gamma power were higher for pictures compared to words. These findings support the idea of a theta-gamma code in binding visual perceptual features during encoding. Furthermore, alpha suppression likely reflects perceptual gating processes in semantic networks and is insensitive to mnemonic and associative binding processes. Gamma oscillations may promote visual perceptual information in visual cortical networks, which is integrated into existing representations by prefrontal control processes, working at a theta pace.

Navigation in Real-World Environments: New Opportunities Afforded by Advances in Mobile Brain Imaging

A central question in neuroscience and psychology is how the mammalian brain represents the outside world and enables interaction with it. Significant progress on this question has been made in the domain of spatial cognition, where a consistent network of brain regions that represent external space has been identified in both humans and rodents. In rodents, much of the work to date has been done in situations where the animal is free to move about naturally. By contrast, the majority of work carried out to date in humans is static, due to limitations imposed by traditional laboratory based imaging techniques. In recent years, significant progress has been made in bridging the gap between animal and human work by employing virtual reality (VR) technology to simulate aspects of real-world navigation. Despite this progress, the VR studies often fail to fully simulate important aspects of real-world navigation, where information derived from self-motion is integrated with representations of environmental features and task goals. In the current review article, we provide a brief overview of animal and human imaging work to date, focusing on commonalties and differences in findings across species. Following on from this we discuss VR studies of spatial cognition, outlining limitations and developments, before introducing mobile brain imaging techniques and describe technical challenges and solutions for real-world recording. Finally, we discuss how these advances in mobile brain imaging technology, provide an unprecedented opportunity to illuminate how the brain represents complex multifaceted information during naturalistic navigation.

Modulation of Neural Activity during Guided Viewing of Visual Art

Mobile Brain-Body Imaging (MoBI) technology was deployed to record multi-modal data from 209 participants to examine the brain's response to artistic stimuli at the Museo de Arte Contemporáneo (MARCO) in Monterrey, México. EEG signals were recorded as the subjects walked through the exhibit in guided groups of 6-8 people. Moreover, guided groups were either provided with an explanation of each art piece (Guided-E), or given no explanation (Guided-NE). The study was performed using portable Muse (InteraXon, Inc, Toronto, ON, Canada) headbands with four dry electrodes located at AF7, AF8, TP9, and TP10. Each participant performed a baseline (BL) control condition devoid of artistic stimuli and selected his/her favorite piece of art (FP) during the guided tour. In this study, we report data related to participants' demographic information and aesthetic preference as well as effects of art viewing on neural activity (EEG) in a select subgroup of 18-30 year-old subjects (Nc = 25) that generated high-quality EEG signals, on both BL and FP conditions. Dependencies on gender, sensor placement, and presence or absence of art explanation were also analyzed. After denoising, clustering of spectral EEG models was used to identify neural patterns associated with BL and FP conditions. Results indicate statistically significant suppression of beta band frequencies (15-25 Hz) in the prefrontal electrodes (AF7 and AF8) during appreciation of subjects' favorite painting, compared to the BL condition, which was significantly different from EEG responses to non-favorite paintings (NFP). No significant differences in brain activity in relation to the presence or absence of explanation during exhibit tours were found. Moreover, a frontal to posterior asymmetry in neural activity was observed, for both BL and FP conditions. These findings provide new information about frequency-related effects of preferred art viewing in brain activity, and support the view that art appreciation is independent of the artists' intent or original interpretation and related to the individual message that viewers themselves provide to each piece.

EEG Recording and Online Signal Processing on Android: A Multiapp Framework for Brain-Computer Interfaces on Smartphone

OBJECTIVE

Our aim was the development and validation of a modular signal processing and classification application enabling online electroencephalography (EEG) signal processing on off-the-shelf mobile Android devices. The software application SCALA (Signal ProCessing and CLassification on Android) supports a standardized communication interface to exchange information with external software and hardware.

APPROACH

In order to implement a closed-loop brain-computer interface (BCI) on the smartphone, we used a multiapp framework, which integrates applications for stimulus presentation, data acquisition, data processing, classification, and delivery of feedback to the user.

MAIN RESULTS

We have implemented the open source signal processing application SCALA. We present timing test results supporting sufficient temporal precision of audio events. We also validate SCALA with a well-established auditory selective attention paradigm and report above chance level classification results for all participants. Regarding the 24-channel EEG signal quality, evaluation results confirm typical sound onset auditory evoked potentials as well as cognitive event-related potentials that differentiate between correct and incorrect task performance feedback.

SIGNIFICANCE

We present a fully smartphone-operated, modular closed-loop BCI system that can be combined with different EEG amplifiers and can easily implement other paradigms.

Neural Correlates of Single- and Dual-Task Walking in the Real World

Recent developments in mobile brain-body imaging (MoBI) technologies have enabled studies of human locomotion where subjects are able to move freely in more ecologically valid scenarios. In this study, MoBI was employed to describe the behavioral and neurophysiological aspects of three different commonly occurring walking conditions in healthy adults. The experimental conditions were self-paced walking, walking while conversing with a friend and lastly walking while texting with a smartphone. We hypothesized that gait performance would decrease with increased cognitive demands and that condition-specific neural activation would involve condition-specific brain areas. Gait kinematics and high density electroencephalography (EEG) were recorded whilst walking around a university campus. Conditions with dual tasks were accompanied by decreased gait performance. Walking while conversing was associated with an increase of theta (θ) and beta (β) neural power in electrodes located over left-frontal and right parietal regions, whereas walking while texting was associated with a decrease of β neural power in a cluster of electrodes over the frontal-premotor and sensorimotor cortices when compared to walking whilst conversing. In conclusion, the behavioral “signatures” of common real-life activities performed outside the laboratory environment were accompanied by differing frequency-specific neural “biomarkers”. The current findings encourage the study of the neural biomarkers of disrupted gait control in neurologically impaired patients.

A Closer Look at the Hippocampus and Memory

Current interpretations of hippocampal memory function are blind to the fact that viewing behaviors are pervasive and complicate the relationships among perception, behavior, memory, and brain activity. For example, hippocampal activity and associative memory demands increase with stimulus complexity. Stimulus complexity also strongly modulates viewing. Associative processing and viewing thus are often confounded, rendering interpretation of hippocampal activity ambiguous. Similar considerations challenge many accounts of hippocampal function. To explain relationships between memory and viewing, we propose that the hippocampus supports the online memory demands necessary to guide visual exploration. The hippocampus thus orchestrates memory-guided exploration that unfolds over time to build coherent memories. This new perspective on hippocampal function harmonizes with the fact that memory formation and exploratory viewing are tightly intertwined.

Spatial Mnemonic Encoding: Theta Power Decreases and Medial Temporal Lobe BOLD Increases Co-Occur during the Usage of the Method of Loci

The method of loci is one, if not the most, efficient mnemonic encoding strategy. This spatial mnemonic combines the core cognitive processes commonly linked to medial temporal lobe (MTL) activity: spatial and associative memory processes. During such processes, fMRI studies consistently demonstrate MTL activity, while electrophysiological studies have emphasized the important role of theta oscillations (3-8 Hz) in the MTL. However, it is still unknown whether increases or decreases in theta power co-occur with increased BOLD signal in the MTL during memory encoding. To investigate this question, we recorded EEG and fMRI separately, while human participants used the spatial method of loci or the pegword method, a similarly associative but nonspatial mnemonic. The more effective spatial mnemonic induced a pronounced theta power decrease source localized to the left MTL compared with the nonspatial associative mnemonic strategy. This effect was mirrored by BOLD signal increases in the MTL. Successful encoding, irrespective of the strategy used, elicited decreases in left temporal theta power and increases in MTL BOLD activity. This pattern of results suggests a negative relationship between theta power and BOLD signal changes in the MTL during memory encoding and spatial processing. The findings extend the well known negative relation of alpha/beta oscillations and BOLD signals in the cortex to theta oscillations in the MTL.