Hierarchical Event Descriptors (HED): Semi-Structured Tagging for Real-World Events in Large-Scale EEG

Open Data In Neurophysiology: Advancements, Solutions & Challenges

Across the life sciences, an ongoing effort over the last 50 years has made data and methods more reproducible and transparent. This openness has led to transformative insights and vastly accelerated scientific progress1,2. For example, structural biology3 and genomics4,5 have undertaken systematic collection and publication of protein sequences and structures over the past half-century, and these data have led to scientific breakthroughs that were unthinkable when data collection first began (e.g.6). We believe that neuroscience is poised to follow the same path, and that principles of open data and open science will transform our understanding of the nervous system in ways that are impossible to predict at the moment. To this end, new social structures along with active and open scientific communities are essential7 to facilitate and expand the still limited adoption of open science practices in our field8. Unified by shared values of openness, we set out to organize a symposium for Open Data in Neuroscience (ODIN) to strengthen our community and facilitate transformative neuroscience research at large. In this report, we share what we learned during this first ODIN event. We also lay out plans for how to grow this movement, document emerging conversations, and propose a path toward a better and more transparent science of tomorrow.

EPAT: a user-friendly MATLAB toolbox for EEG/ERP data processing and analysis

Background

At the intersection of neural monitoring and decoding, event-related potential (ERP) based on electroencephalography (EEG) has opened a window into intrinsic brain function. The stability of ERP makes it frequently employed in the field of neuroscience. However, project-specific custom code, tracking of user-defined parameters, and the large diversity of commercial tools have limited clinical application.

Methods

We introduce an open-source, user-friendly, and reproducible MATLAB toolbox named EPAT that includes a variety of algorithms for EEG data preprocessing. It provides EEGLAB-based template pipelines for advanced multi-processing of EEG, magnetoencephalography, and polysomnogram data. Participants evaluated EEGLAB and EPAT across 14 indicators, with satisfaction ratings analyzed using the Wilcoxon signed-rank test or paired t-test based on distribution normality.

Results

EPAT eases EEG signal browsing and preprocessing, EEG power spectrum analysis, independent component analysis, time-frequency analysis, ERP waveform drawing, and topological analysis of scalp voltage. A user-friendly graphical user interface allows clinicians and researchers with no programming background to use EPAT.

Conclusion

This article describes the architecture, functionalities, and workflow of the toolbox. The release of EPAT will help advance EEG methodology and its application to clinical translational studies.

FAIR human neuroscientific data sharing to advance AI driven research and applications: Legal frameworks and missing metadata standards

Modern AI supported research holds many promises for basic and applied science. However, the application of AI methods is often limited because most labs cannot, on their own, acquire large and diverse datasets, which are best for training these methods. Data sharing and open science initiatives promise some relief to the problem, but only if the data are provided in a usable way. The FAIR principles state very general requirements for useful data sharing: they should be findable, accessible, interoperable, and reusable. This article will focus on two challenges to implement the FAIR framework for human neuroscience data. On the one hand, human data can fall under special legal protection. The legal frameworks regulating how and what data can be openly shared differ greatly across countries which can complicate data sharing or even discourage researchers from doing so. Moreover, openly accessible data require standardization of data and metadata organization and annotation in order to become interpretable and useful. This article briefly introduces open neuroscience initiatives that support the implementation of the FAIR principles. It then reviews legal frameworks, their consequences for accessibility of human neuroscientific data and some ethical implications. We hope this comparison of legal jurisdictions helps to elucidate that some alleged obstacles for data sharing only require an adaptation of procedures but help to protect the privacy of our most generous donors to research … our study participants. Finally, it elaborates on the problem of missing standards for metadata annotation and introduces initiatives that aim at developing tools to make neuroscientific data acquisition and analysis pipelines FAIR by design. While the paper focuses on making human neuroscience data useful for data-intensive AI the general considerations hold for other fields where large amounts of openly available human data would be helpful.

Learning neural decoders without labels using multiple data streams

OBJECTIVE

Recent advances in neural decoding have accelerated the development of brain-computer interfaces aimed at assisting users with everyday tasks such as speaking, walking, and manipulating objects. However, current approaches for training neural decoders commonly require large quantities of labeled data, which can be laborious or infeasible to obtain in real-world settings. Alternatively, self-supervised models that share self-generated pseudo-labels between two data streams have shown exceptional performance on unlabeled audio and video data, but it remains unclear how well they extend to neural decoding.

APPROACH

We learn neural decoders without labels by leveraging multiple simultaneously recorded data streams, including neural, kinematic, and physiological signals. Specifically, we apply cross-modal, self-supervised deep clustering to train decoders that can classify movements from brain recordings. After training, we then isolate the decoders for each input data stream and compare the accuracy of decoders trained using cross-modal deep clustering against supervised and unimodal, self-supervised models.

MAIN RESULTS

We find that sharing pseudo-labels between two data streams during training substantially increases decoding performance compared to unimodal, self-supervised models, with accuracies approaching those of supervised decoders trained on labeled data. Next, we extend cross-modal decoder training to three or more modalities, achieving state-of-the-art neural decoding accuracy that matches or slightly exceeds the performance of supervised models. Significance: We demonstrate that cross-modal, self-supervised decoding can be applied to train neural decoders when few or no labels are available and extend the cross-modal framework to share information among three or more data streams, further improving self-supervised training.

Building FAIR Functionality: Annotating Events in Time Series Data Using Hierarchical Event Descriptors (HED)

Human electrophysiological and related time series data are often acquired in complex, event-rich environments. However, the resulting recorded brain or other dynamics are often interpreted in relation to more sparsely recorded or subsequently-noted events. Currently a substantial gap exists between the level of event description required by current digital data archiving standards and the level of annotation required for successful analysis of event-related data across studies, environments, and laboratories. Manifold challenges must be addressed, most prominently ontological clarity, vocabulary extensibility, annotation tool availability, and overall usability, to allow and promote sharing of data with an effective level of descriptive detail for labeled events. Motivating data authors to perform the work needed to adequately annotate their data is a key challenge. This paper describes new developments in the Hierarchical Event Descriptor (HED) system for addressing these issues. We recap the evolution of HED and its acceptance by the Brain Imaging Data Structure (BIDS) movement, describe the recent release of HED-3G, a third generation HED tools and design framework, and discuss directions for future development. Given consistent, sufficiently detailed, tool-enabled, field-relevant annotation of the nature of recorded events, prospects are bright for large-scale analysis and modeling of aggregated time series data, both in behavioral and brain imaging sciences and beyond.

Dementia ConnEEGtome: Towards multicentric harmonization of EEG connectivity in neurodegeneration

The proposal to use brain connectivity as a biomarker for dementia phenotyping can be potentiated by conducting large-scale multicentric studies using high-density electroencephalography (hd- EEG). Nevertheless, several barriers preclude the development of a systematic "ConnEEGtome" in dementia research. Here we review critical sources of variability in EEG connectivity studies, and provide general guidelines for multicentric protocol harmonization. We describe how results can be impacted by the choice for data acquisition, and signal processing workflows. The implementation of a particular processing pipeline is conditional upon assumptions made by researchers about the nature of EEG. Due to these assumptions, EEG connectivity metrics are typically applicable to restricted scenarios, e.g., to a particular neurocognitive disorder. "Ground truths" for the choice of processing workflow and connectivity analysis are impractical. Consequently, efforts should be directed to harmonizing experimental procedures, data acquisition, and the first steps of the preprocessing pipeline. Conducting multiple analyses of the same data and a proper integration of the results need to be considered in additional processing steps. Furthermore, instead of using a single connectivity measure, using a composite metric combining different connectivity measures brings a powerful strategy to scale up the replicability of multicentric EEG connectivity studies. These composite metrics can boost the predictive strength of diagnostic tools for dementia. Moreover, the implementation of multi-feature machine learning classification systems that include EEG-based connectivity analyses may help to exploit the potential of multicentric studies combining clinical-cognitive, molecular, genetics, and neuroimaging data towards a multi-dimensional characterization of the dementia.

Capturing the nature of events and event context using hierarchical event descriptors (HED)

Event-related data analysis plays a central role in EEG and MEG (MEEG) and other neuroimaging modalities including fMRI. Choices about which events to report and how to annotate their full natures significantly influence the value, reliability, and reproducibility of neuroimaging datasets for further analysis and meta- or mega-analysis. A powerful annotation strategy using the new third-generation formulation of the Hierarchical Event Descriptors (HED) framework and tools (hedtags.org) combines robust event description with details of experiment design and metadata in a human-readable as well as machine-actionable form, making event annotation relevant to the full range of neuroimaging and other time series data. This paper considers the event design and annotation process using as a case study the well-known multi-subject, multimodal dataset of Wakeman and Henson made available by its authors as a Brain Imaging Data Structure (BIDS) dataset (bids.neuroimaging.io). We propose a set of best practices and guidelines for event annotation integrated in a natural way into the BIDS metadata file architecture, examine the impact of event design decisions, and provide a working example of organizing events in MEEG and other neuroimaging data. We demonstrate how annotations using HED can document events occurring during neuroimaging experiments as well as their interrelationships, providing machine-actionable annotation enabling automated within- and across-experiment analysis and comparisons. We discuss the evolution of HED software tools and have made available an accompanying HED-annotated BIDS-formated edition of the MEEG data of the Wakeman and Henson dataset (openneuro.org, ds003645).

The open EEGLAB portal Interface: High-Performance computing with EEGLAB

EEGLAB signal processing environment is currently the leading open-source software for processing electroencephalographic (EEG) data. The Neuroscience Gateway (NSG, nsgportal.org) is a web and API-based portal allowing users to easily run a variety of neuroscience-related software on high-performance computing (HPC) resources in the U.S. XSEDE network. We have reported recently (Delorme et al., 2019) on the Open EEGLAB Portal expansion of the free NSG services to allow the neuroscience community to build and run MATLAB pipelines using the EEGLAB tool environment. We are now releasing an EEGLAB plug-in, nsgportal, that interfaces EEGLAB with NSG directly from within EEGLAB running on MATLAB on any personal lab computer. The plug-in features a flexible MATLAB graphical user interface (GUI) that allows users to easily submit, interact with, and manage NSG jobs, and to retrieve and examine their results. Command line nsgportal tools supporting these GUI functionalities allow EEGLAB users and plug-in tool developers to build largely automated functions and workflows that include optional NSG job submission and processing. Here we present details on nsgportal implementation and documentation, provide user tutorials on example applications, and show sample test results comparing computation times using HPC versus laptop processing.

How Sensitive Are EEG Results to Preprocessing Methods: A Benchmarking Study

Although several guidelines for best practices in EEG preprocessing have been released, even studies that strictly adhere to those guidelines contain considerable variation in the ways that the recommended methods are applied. An open question for researchers is how sensitive the results of EEG analyses are to variations in preprocessing methods and parameters. To address this issue, we analyze the effect of preprocessing methods on downstream EEG analysis using several simple signal and event-related measures. Signal measures include recording-level channel amplitudes, study-level channel amplitude dispersion, and recording spectral characteristics. Event-related methods include ERPs and ERSPs and their correlations across methods for a diverse set of stimulus events. Our analysis also assesses differences in residual signals both in the time and spectral domains after blink artifacts have been removed. Using fully automated pipelines, we evaluate these measures across 17 EEG studies for two ICA-based preprocessing approaches (LARG, MARA) plus two variations of Artifact Subspace Reconstruction (ASR). Although the general structure of the results is similar across these preprocessing methods, there are significant differences, particularly in the low-frequency spectral features and in the residuals left by blinks. These results argue for detailed reporting of processing details as suggested by most guidelines, but also for using a federation of automated processing pipelines and comparison tools to quantify effects of processing choices as part of the research reporting.

Workshops of the Seventh International Brain-Computer Interface Meeting: Not Getting Lost in Translation

The Seventh International Brain-Computer Interface (BCI) Meeting was held May 21-25th, 2018 at the Asilomar Conference Grounds, Pacific Grove, California, United States. The interactive nature of this conference was embodied by 25 workshops covering topics in BCI (also called brain-machine interface) research. Workshops covered foundational topics such as hardware development and signal analysis algorithms, new and imaginative topics such as BCI for virtual reality and multi-brain BCIs, and translational topics such as clinical applications and ethical assumptions of BCI development. BCI research is expanding in the diversity of applications and populations for whom those applications are being developed. BCI applications are moving toward clinical readiness as researchers struggle with the practical considerations to make sure that BCI translational efforts will be successful. This paper summarizes each workshop, providing an overview of the topic of discussion, references for additional information, and identifying future issues for research and development that resulted from the interactions and discussion at the workshop.

High-density EEG mobile brain/body imaging data recorded during a challenging auditory gait pacing task

In this report we present a mobile brain/body imaging (MoBI) dataset that allows study of source-resolved cortical dynamics supporting coordinated gait movements in a rhythmic auditory cueing paradigm. Use of an auditory pacing stimulus stream has been recommended to identify deficits and treat gait impairments in neurologic populations. Here, the rhythmic cueing paradigm required healthy young participants to walk on a treadmill (constant speed) while attempting to maintain step synchrony with an auditory pacing stream and to adapt their step length and rate to unanticipated shifts in tempo of the pacing stimuli (e.g., sudden shifts to a faster or slower tempo). High-density electroencephalography (EEG, 108 channels), surface electromyography (EMG, bilateral tibialis anterior), pressure sensors on the heel (to register timing of heel strikes), and goniometers (knee, hip, and ankle joint angles) were concurrently recorded in 20 participants. The data is provided in the Brain Imaging Data Structure (BIDS) format to promote data sharing and reuse, and allow the inclusion of the data into fully automated data analysis workflows.

EEG-BIDS, an extension to the brain imaging data structure for electroencephalography

The Brain Imaging Data Structure (BIDS) project is a rapidly evolving effort in the human brain imaging research community to create standards allowing researchers to readily organize and share study data within and between laboratories. Here we present an extension to BIDS for electroencephalography (EEG) data, EEG-BIDS, along with tools and references to a series of public EEG datasets organized using this new standard.

SEREEGA: Simulating event-related EEG activity

BACKGROUND

Electroencephalography (EEG) is a popular method to monitor brain activity, but it is difficult to evaluate EEG-based analysis methods because no ground-truth brain activity is available for comparison. Therefore, in order to test and evaluate such methods, researchers often use simulated EEG data instead of actual EEG recordings. Simulated data can be used, among other things, to assess or compare signal processing and machine learning algorithms, to model EEG variabilities, and to design source reconstruction methods.

NEW METHOD

We present SEREEGA, Simulating Event-Related EEG Activity. SEREEGA is a free and open-source MATLAB-based toolbox dedicated to the generation of simulated epochs of EEG data. It is modular and extensible, at initial release supporting five different publicly available head models and capable of simulating multiple different types of signals mimicking brain activity. This paper presents the architecture and general workflow of this toolbox, as well as a simulated data set demonstrating some of its functions. The toolbox is available at https://github.com/lrkrol/SEREEGA.

RESULTS

The simulated data allows established analysis pipelines and classification methods to be applied and is capable of producing realistic results.

COMPARISON WITH EXISTING METHODS

Most simulated EEG is coded from scratch. The few open-source methods in existence focus on specific applications or signal types, such as connectivity. SEREEGA unifies the majority of past simulation methods reported in the literature into one toolbox.

CONCLUSION

SEREEGA is a general-purpose toolbox to simulate ground-truth EEG data.

An 18-subject EEG data collection using a visual-oddball task, designed for benchmarking algorithms and headset performance comparisons

This data note describes an 18-subject EEG (electroencephalogram) data collection from an experiment in which subjects performed a standard visual oddball task. Several research projects have used this data to test artifact detection, classification, transfer learning, EEG preprocessing, blink detection, and automated annotation algorithms. We are releasing the data in three formats to enable benchmarking of EEG algorithms in many areas. The data was acquired using a Biosemi Active 2 EEG headset and includes 64 channels of EEG, 4 channels of EOG (electrooculogram), and 2 mastoid reference channels.

A Review of Issues Related to Data Acquisition and Analysis in EEG/MEG Studies

Electroencephalography (EEG) and magnetoencephalography (MEG) are non-invasive electrophysiological methods, which record electric potentials and magnetic fields due to electric currents in synchronously-active neurons. With MEG being more sensitive to neural activity from tangential currents and EEG being able to detect both radial and tangential sources, the two methods are complementary. Over the years, neurophysiological studies have changed considerably: high-density recordings are becoming de rigueur; there is interest in both spontaneous and evoked activity; and sophisticated artifact detection and removal methods are available. Improved head models for source estimation have also increased the precision of the current estimates, particularly for EEG and combined EEG/MEG. Because of their complementarity, more investigators are beginning to perform simultaneous EEG/MEG studies to gain more complete information about neural activity. Given the increase in methodological complexity in EEG/MEG, it is important to gather data that are of high quality and that are as artifact free as possible. Here, we discuss some issues in data acquisition and analysis of EEG and MEG data. Practical considerations for different types of EEG and MEG studies are also discussed.