1
|
Xue C, Chen Y, Thompson WF, Liu F, Jiang C. Time-varying similarity of neural responses to musical tension is shaped by physical features and musical themes. Int J Psychophysiol 2024; 202:112387. [PMID: 38909958 DOI: 10.1016/j.ijpsycho.2024.112387] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2024] [Revised: 05/20/2024] [Accepted: 06/18/2024] [Indexed: 06/25/2024]
Abstract
The similarity of understanding is important for music experience and communication, but little is understood about the sources of this common knowledge. Although neural responses to the same piece of music are known to be similar across listeners, it remains unclear whether this neural response similarity is linked to musical understanding and the role of dynamic musical attributes in shaping it. Our study addresses this gap by investigating the relationship between neural response similarity, musical tension, and dynamic musical attributes. Using electroencephalography-based inter-subject correlation (EEG-ISC), we examined how the neural response similarity among listeners varies throughout the evaluation of musical tension in the first movement of Beethoven's Piano Sonata No. 8. Participants continuously rated the degree of alignment between musical events and their expectations, while neural activity was recorded using electroencephalography (EEG). The results showed that neural response similarity fluctuated in tandem with musical tension, with increased similarity observed during moments of heightened tension. This time-varying neural response similarity was influenced by two dynamic attributes contributing to musical tension: physical features and musical themes. Specifically, its fluctuation was driven by physical features, and the patterns of its variation were modulated by musical themes, with similar time-varying patterns observed across similar thematic materials. These findings offer valuable insight into the role of dynamic musical attributes in shaping neural response similarity, and reveal an important source and mechanism of shared musical understandings.
Collapse
Affiliation(s)
- Chao Xue
- Department of Psychology, Shanghai Normal University, Shanghai 200234, China
| | - Yiran Chen
- Research Institute of the McGill University Health Centre, Montreal, QC, Canada
| | | | - Fang Liu
- School of Psychology and Clinical Language Sciences, University of Reading, Reading RG6 6AL, UK
| | - Cunmei Jiang
- Music College, Shanghai Normal University, Shanghai 200234, China.
| |
Collapse
|
2
|
Kaneshiro B, Nguyen DT, Norcia AM, Dmochowski JP, Berger J. Inter-subject correlation of electroencephalographic and behavioural responses reflects time-varying engagement with natural music. Eur J Neurosci 2024; 59:3162-3183. [PMID: 38626924 DOI: 10.1111/ejn.16324] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2021] [Revised: 02/28/2024] [Accepted: 03/07/2024] [Indexed: 06/15/2024]
Abstract
Musical engagement can be conceptualized through various activities, modes of listening and listener states. Recent research has reported that a state of focused engagement can be indexed by the inter-subject correlation (ISC) of audience responses to a shared naturalistic stimulus. While statistically significant ISC has been reported during music listening, we lack insight into the temporal dynamics of engagement over the course of musical works-such as those composed in the Western classical style-which involve the formulation of expectations that are realized or derailed at subsequent points of arrival. Here, we use the ISC of electroencephalographic (EEG) and continuous behavioural (CB) responses to investigate the time-varying dynamics of engagement with functional tonal music. From a sample of adult musicians who listened to a complete cello concerto movement, we found that ISC varied throughout the excerpt for both measures. In particular, significant EEG ISC was observed during periods of musical tension that built to climactic highpoints, while significant CB ISC corresponded more to declarative entrances and points of arrival. Moreover, we found that a control stimulus retaining envelope characteristics of the intact music, but little other temporal structure, also elicited significantly correlated EEG and CB responses, though to lesser extents than the original version. In sum, these findings shed light on the temporal dynamics of engagement during music listening and clarify specific aspects of musical engagement that may be indexed by each measure.
Collapse
Affiliation(s)
- Blair Kaneshiro
- Center for Computer Research in Music and Acoustics, Stanford University, Stanford, California, USA
- Center for the Study of Language and Information, Stanford University, Stanford, California, USA
- Graduate School of Education, Stanford University, Stanford, California, USA
| | - Duc T Nguyen
- Center for Computer Research in Music and Acoustics, Stanford University, Stanford, California, USA
- Center for the Study of Language and Information, Stanford University, Stanford, California, USA
| | - Anthony M Norcia
- Department of Psychology, Stanford University, Stanford, California, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, California, USA
| | - Jacek P Dmochowski
- Department of Biomedical Engineering, City College of New York, New York, New York, USA
| | - Jonathan Berger
- Center for Computer Research in Music and Acoustics, Stanford University, Stanford, California, USA
| |
Collapse
|
3
|
Yao Y, Stebner A, Tuytelaars T, Geirnaert S, Bertrand A. Identifying temporal correlations between natural single-shot videos and EEG signals. J Neural Eng 2024; 21:016018. [PMID: 38277701 DOI: 10.1088/1741-2552/ad2333] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2023] [Accepted: 01/26/2024] [Indexed: 01/28/2024]
Abstract
Objective.Electroencephalography (EEG) is a widely used technology for recording brain activity in brain-computer interface (BCI) research, where understanding the encoding-decoding relationship between stimuli and neural responses is a fundamental challenge. Recently, there is a growing interest in encoding-decoding natural stimuli in a single-trial setting, as opposed to traditional BCI literature where multi-trial presentations of synthetic stimuli are commonplace. While EEG responses to natural speech have been extensively studied, such stimulus-following EEG responses to natural video footage remain underexplored.Approach.We collect a new EEG dataset with subjects passively viewing a film clip and extract a few video features that have been found to be temporally correlated with EEG signals. However, our analysis reveals that these correlations are mainly driven by shot cuts in the video. To avoid the confounds related to shot cuts, we construct another EEG dataset with natural single-shot videos as stimuli and propose a new set of object-based features.Main results.We demonstrate that previous video features lack robustness in capturing the coupling with EEG signals in the absence of shot cuts, and that the proposed object-based features exhibit significantly higher correlations. Furthermore, we show that the correlations obtained with these proposed features are not dominantly driven by eye movements. Additionally, we quantitatively verify the superiority of the proposed features in a match-mismatch task. Finally, we evaluate to what extent these proposed features explain the variance in coherent stimulus responses across subjects.Significance.This work provides valuable insights into feature design for video-EEG analysis and paves the way for applications such as visual attention decoding.
Collapse
Affiliation(s)
- Yuanyuan Yao
- Department of Electrical Engineering, STADIUS, KU Leuven, Leuven, Belgium
| | - Axel Stebner
- Department of Electrical Engineering, PSI, KU Leuven, Leuven, Belgium
| | - Tinne Tuytelaars
- Department of Electrical Engineering, PSI, KU Leuven, Leuven, Belgium
| | - Simon Geirnaert
- Department of Electrical Engineering, STADIUS, Department of Neurosciences, ExpORL, KU Leuven, Leuven, Belgium
| | - Alexander Bertrand
- Department of Electrical Engineering, STADIUS, KU Leuven, Leuven, Belgium
| |
Collapse
|
4
|
Sanaullah, Koravuna S, Rückert U, Jungeblut T. Evaluation of Spiking Neural Nets-Based Image Classification Using the Runtime Simulator RAVSim. Int J Neural Syst 2023; 33:2350044. [PMID: 37604777 DOI: 10.1142/s0129065723500442] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/23/2023]
Abstract
Spiking Neural Networks (SNNs) help achieve brain-like efficiency and functionality by building neurons and synapses that mimic the human brain's transmission of electrical signals. However, optimal SNN implementation requires a precise balance of parametric values. To design such ubiquitous neural networks, a graphical tool for visualizing, analyzing, and explaining the internal behavior of spikes is crucial. Although some popular SNN simulators are available, these tools do not allow users to interact with the neural network during simulation. To this end, we have introduced the first runtime interactive simulator, called Runtime Analyzing and Visualization Simulator (RAVSim),a developed to analyze and dynamically visualize the behavior of SNNs, allowing end-users to interact, observe output concentration reactions, and make changes directly during the simulation. In this paper, we present RAVSim with the current implementation of runtime interaction using the LIF neural model with different connectivity schemes, an image classification model using SNNs, and a dataset creation feature. Our main objective is to primarily investigate binary classification using SNNs with RGB images. We created a feed-forward network using the LIF neural model for an image classification algorithm and evaluated it by using RAVSim. The algorithm classifies faces with and without masks, achieving an accuracy of 91.8% using 1000 neurons in a hidden layer, 0.0758 MSE, and an execution time of ∼10[Formula: see text]min on the CPU. The experimental results show that using RAVSim not only increases network design speed but also accelerates user learning capability.
Collapse
Affiliation(s)
- Sanaullah
- Department of Engineering and Mathematics, Bielefeld University of Applied Science, Bielefeld, Germany
| | - Shamini Koravuna
- Department of Cognitive Interaction Technology Center, Bielefeld University, Bielefeld, Germany
| | - Ulrich Rückert
- Department of Cognitive Interaction Technology Center, Bielefeld University, Bielefeld, Germany
| | - Thorsten Jungeblut
- Department of Engineering and Mathematics, Bielefeld University of Applied Science, Bielefeld, Germany
| |
Collapse
|
5
|
Ueno F, Shimada S. Inter-subject correlations of EEG reflect subjective arousal and acoustic features of music. Front Hum Neurosci 2023; 17:1225377. [PMID: 37671247 PMCID: PMC10475548 DOI: 10.3389/fnhum.2023.1225377] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Accepted: 07/31/2023] [Indexed: 09/07/2023] Open
Abstract
Background Research on music-induced emotion and brain activity is constantly expanding. Although studies using inter-subject correlation (ISC), a collectively shared brain activity analysis method, have been conducted, whether ISC during music listening represents the music preferences of a large population remains uncertain; additionally, it remains unclear which factors influence ISC during music listening. Therefore, here, we aimed to investigate whether the ISCs of electroencephalography (EEG) during music listening represent a preference for music reflecting engagement or interest of a large population in music. Methods First, we selected 21 pieces of music from the Billboard Japan Hot 100 chart of 2017, which served as an indicator of preference reflecting the engagement and interest of a large population. To ensure even representation, we chose one piece for every fifth song on the chart, spanning from highly popular music to less popular ones. Next, we recorded EEG signals while the subjects listened to the selected music, and they were asked to evaluate four aspects (preference, enjoyment, frequency of listening, and arousal) for each song. Subsequently, we conducted ISC analysis by utilizing the first three principal components of EEG, which were highly correlated across subjects and extracted through correlated component analysis (CorrCA). We then explored whether music with high preferences that reflected the engagement and interest of large population had high ISC values. Additionally, we employed cluster analysis on all 21 pieces of music, utilizing the first three principal components of EEG, to investigate the impact of emotions and musical characteristics on EEG ISC during music listening. Results A significant distinction was noted between the mean ISC values of the 10 higher-ranked pieces of music compared to the 10 lower-ranked pieces of music [t(542) = -1.97, p = 0.0025]. This finding suggests that ISC values may correspond preferences reflecting engagement or interest of a large population. Furthermore, we found that significant variations were observed in the first three principal component values among the three clusters identified through cluster analysis, along with significant differences in arousal levels. Moreover, the characteristics of the music (tonality and tempo) differed among the three clusters. This indicates that the principal components, which exhibit high correlation among subjects and were employed in calculating ISC values, represent both subjects' arousal levels and specific characteristics of the music. Conclusion Subjects' arousal values during music listening and music characteristics (tonality and tempo) affect ISC values, which represent the interest of a large population in music.
Collapse
Affiliation(s)
- Fuyu Ueno
- Department of Electronics and Bioinformatics, School of Science and Technology, Meiji University, Kawasaki, Japan
- Japan Society for the Promotion of Science, Tokyo, Japan
| | - Sotaro Shimada
- Department of Electronics and Bioinformatics, School of Science and Technology, Meiji University, Kawasaki, Japan
| |
Collapse
|
6
|
Nentwich M, Leszczynski M, Russ BE, Hirsch L, Markowitz N, Sapru K, Schroeder CE, Mehta AD, Bickel S, Parra LC. Semantic novelty modulates neural responses to visual change across the human brain. Nat Commun 2023; 14:2910. [PMID: 37217478 PMCID: PMC10203305 DOI: 10.1038/s41467-023-38576-5] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2022] [Accepted: 05/08/2023] [Indexed: 05/24/2023] Open
Abstract
Our continuous visual experience in daily life is dominated by change. Previous research has focused on visual change due to stimulus motion, eye movements or unfolding events, but not their combined impact across the brain, or their interactions with semantic novelty. We investigate the neural responses to these sources of novelty during film viewing. We analyzed intracranial recordings in humans across 6328 electrodes from 23 individuals. Responses associated with saccades and film cuts were dominant across the entire brain. Film cuts at semantic event boundaries were particularly effective in the temporal and medial temporal lobe. Saccades to visual targets with high visual novelty were also associated with strong neural responses. Specific locations in higher-order association areas showed selectivity to either high or low-novelty saccades. We conclude that neural activity associated with film cuts and eye movements is widespread across the brain and is modulated by semantic novelty.
Collapse
Affiliation(s)
- Maximilian Nentwich
- Department of Biomedical Engineering, The City College of New York, New York, NY, USA.
| | - Marcin Leszczynski
- Departments of Psychiatry and Neurology, Columbia University College of Physicians and Surgeons, New York, NY, USA
- Translational Neuroscience Lab Division, Center for Biomedical Imaging and Neuromodulation, Nathan Kline Institute, Orangeburg, NY, USA
- Cognitive Science Department, Institute of Philosophy, Jagiellonian University, Kraków, Poland
| | - Brian E Russ
- Translational Neuroscience Lab Division, Center for Biomedical Imaging and Neuromodulation, Nathan Kline Institute, Orangeburg, NY, USA
- Nash Family Department of Neuroscience and Friedman Brain Institute, Icahn School of Medicine, New York, NY, USA
- Department of Psychiatry, New York University at Langone, New York, NY, USA
| | - Lukas Hirsch
- Department of Biomedical Engineering, The City College of New York, New York, NY, USA
| | - Noah Markowitz
- The Feinstein Institutes for Medical Research, Northwell Health, Manhasset, NY, USA
| | - Kaustubh Sapru
- Department of Biomedical Engineering, The City College of New York, New York, NY, USA
| | - Charles E Schroeder
- Departments of Psychiatry and Neurology, Columbia University College of Physicians and Surgeons, New York, NY, USA
- Translational Neuroscience Lab Division, Center for Biomedical Imaging and Neuromodulation, Nathan Kline Institute, Orangeburg, NY, USA
| | - Ashesh D Mehta
- The Feinstein Institutes for Medical Research, Northwell Health, Manhasset, NY, USA
- Departments of Neurosurgery and Neurology, Zucker School of Medicine at Hofstra/Northwell, Manhasset, NY, USA
| | - Stephan Bickel
- Translational Neuroscience Lab Division, Center for Biomedical Imaging and Neuromodulation, Nathan Kline Institute, Orangeburg, NY, USA
- The Feinstein Institutes for Medical Research, Northwell Health, Manhasset, NY, USA
- Departments of Neurosurgery and Neurology, Zucker School of Medicine at Hofstra/Northwell, Manhasset, NY, USA
| | - Lucas C Parra
- Department of Biomedical Engineering, The City College of New York, New York, NY, USA.
| |
Collapse
|
7
|
Schmälzle R, Huskey R. Integrating media content analysis, reception analysis, and media effects studies. Front Neurosci 2023; 17:1155750. [PMID: 37179563 PMCID: PMC10173883 DOI: 10.3389/fnins.2023.1155750] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Accepted: 03/28/2023] [Indexed: 05/15/2023] Open
Abstract
Every day, the world of media is at our fingertips, whether it is watching movies, listening to the radio, or browsing online media. On average, people spend over 8 h per day consuming messages from the mass media, amounting to a total lifetime dose of more than 20 years in which conceptual content stimulates our brains. Effects from this flood of information range from short-term attention bursts (e.g., by breaking news features or viral 'memes') to life-long memories (e.g., of one's favorite childhood movie), and from micro-level impacts on an individual's memory, attitudes, and behaviors to macro-level effects on nations or generations. The modern study of media's influence on society dates back to the 1940s. This body of mass communication scholarship has largely asked, "what is media's effect on the individual?" Around the time of the cognitive revolution, media psychologists began to ask, "what cognitive processes are involved in media processing?" More recently, neuroimaging researchers started using real-life media as stimuli to examine perception and cognition under more natural conditions. Such research asks: "what can media tell us about brain function?" With some exceptions, these bodies of scholarship often talk past each other. An integration offers new insights into the neurocognitive mechanisms through which media affect single individuals and entire audiences. However, this endeavor faces the same challenges as all interdisciplinary approaches: Researchers with different backgrounds have different levels of expertise, goals, and foci. For instance, neuroimaging researchers label media stimuli as "naturalistic" although they are in many ways rather artificial. Similarly, media experts are typically unfamiliar with the brain. Neither media creators nor neuroscientifically oriented researchers approach media effects from a social scientific perspective, which is the domain of yet another species. In this article, we provide an overview of approaches and traditions to studying media, and we review the emerging literature that aims to connect these streams. We introduce an organizing scheme that connects the causal paths from media content → brain responses → media effects and discuss network control theory as a promising framework to integrate media content, reception, and effects analyses.
Collapse
Affiliation(s)
- Ralf Schmälzle
- Department of Communication, Michigan State University, East Lansing, MI, United States
| | - Richard Huskey
- Department of Communication, University of California, Davis, Davis, CA, United States
- Cognitive Science Program, University of California, Davis, Davis, CA, United States
- Center for Mind and Brain, University of California, Davis, Davis, CA, United States
| |
Collapse
|
8
|
Affiliation(s)
- Max Dabagia
- School of Computer Science, Georgia Institute of Technology, Atlanta, GA, USA
| | - Konrad P Kording
- Department of Biomedical Engineering, University of Pennsylvania, Philadelphia, PA, USA
| | - Eva L Dyer
- Department of Biomedical Engineering, Georgia Institute of Technology, Atlanta, GA, USA.
| |
Collapse
|
9
|
Lomas JD, Lin A, Dikker S, Forster D, Lupetti ML, Huisman G, Habekost J, Beardow C, Pandey P, Ahmad N, Miyapuram K, Mullen T, Cooper P, van der Maden W, Cross ES. Resonance as a Design Strategy for AI and Social Robots. Front Neurorobot 2022; 16:850489. [PMID: 35574227 PMCID: PMC9097027 DOI: 10.3389/fnbot.2022.850489] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2022] [Accepted: 03/23/2022] [Indexed: 11/20/2022] Open
Abstract
Resonance, a powerful and pervasive phenomenon, appears to play a major role in human interactions. This article investigates the relationship between the physical mechanism of resonance and the human experience of resonance, and considers possibilities for enhancing the experience of resonance within human-robot interactions. We first introduce resonance as a widespread cultural and scientific metaphor. Then, we review the nature of "sympathetic resonance" as a physical mechanism. Following this introduction, the remainder of the article is organized in two parts. In part one, we review the role of resonance (including synchronization and rhythmic entrainment) in human cognition and social interactions. Then, in part two, we review resonance-related phenomena in robotics and artificial intelligence (AI). These two reviews serve as ground for the introduction of a design strategy and combinatorial design space for shaping resonant interactions with robots and AI. We conclude by posing hypotheses and research questions for future empirical studies and discuss a range of ethical and aesthetic issues associated with resonance in human-robot interactions.
Collapse
Affiliation(s)
- James Derek Lomas
- Department of Human Centered Design, Faculty of Industrial Design Engineering, Delft University of Technology, Delft, Netherlands
| | - Albert Lin
- Center for Human Frontiers, Qualcomm Institute, University of California, San Diego, San Diego, CA, United States
| | - Suzanne Dikker
- Department of Psychology, New York University, New York, NY, United States
- Department of Clinical Psychology, Vrije Universiteit Amsterdam, Amsterdam, Netherlands
| | - Deborah Forster
- Center for Human Frontiers, Qualcomm Institute, University of California, San Diego, San Diego, CA, United States
| | - Maria Luce Lupetti
- Department of Human Centered Design, Faculty of Industrial Design Engineering, Delft University of Technology, Delft, Netherlands
| | - Gijs Huisman
- Department of Human Centered Design, Faculty of Industrial Design Engineering, Delft University of Technology, Delft, Netherlands
| | - Julika Habekost
- The Design Lab, California Institute of Information and Communication Technologies, University of California, San Diego, San Diego, CA, United States
| | - Caiseal Beardow
- Department of Human Centered Design, Faculty of Industrial Design Engineering, Delft University of Technology, Delft, Netherlands
| | - Pankaj Pandey
- Centre for Cognitive and Brain Sciences, Indian Institute of Technology, Gandhinagar, India
| | - Nashra Ahmad
- Centre for Cognitive and Brain Sciences, Indian Institute of Technology, Gandhinagar, India
| | - Krishna Miyapuram
- Centre for Cognitive and Brain Sciences, Indian Institute of Technology, Gandhinagar, India
| | - Tim Mullen
- Intheon Labs, San Diego, CA, United States
| | - Patrick Cooper
- Department of Physics, Duquesne University, Pittsburgh, PA, United States
| | - Willem van der Maden
- Department of Human Centered Design, Faculty of Industrial Design Engineering, Delft University of Technology, Delft, Netherlands
| | - Emily S. Cross
- Social Robotics, Institute of Neuroscience and Psychology, School of Computing Science, University of Glasgow, Glasgow, United Kingdom
- SOBA Lab, School of Psychology, Macquarie University, Sydney, NSW, Australia
| |
Collapse
|
10
|
Arefnezhad S, Hamet J, Eichberger A, Frühwirth M, Ischebeck A, Koglbauer IV, Moser M, Yousefi A. Driver drowsiness estimation using EEG signals with a dynamical encoder-decoder modeling framework. Sci Rep 2022; 12:2650. [PMID: 35173189 PMCID: PMC8850607 DOI: 10.1038/s41598-022-05810-x] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2021] [Accepted: 01/14/2022] [Indexed: 01/22/2023] Open
Abstract
Drowsiness is a leading cause of accidents on the road as it negatively affects the driver’s ability to safely operate a vehicle. Neural activity recorded by EEG electrodes is a widely used physiological correlate of driver drowsiness. This paper presents a novel dynamical modeling solution to estimate the instantaneous level of the driver drowsiness using EEG signals, where the PERcentage of eyelid CLOSure (PERCLOS) is employed as the ground truth of driver drowsiness. Applying our proposed modeling framework, we find neural features present in EEG data that encode PERCLOS. In the decoding phase, we use a Bayesian filtering solution to estimate the PERCLOS level over time. A data set that comprises 18 driving tests, conducted by 13 drivers, has been used to investigate the performance of the proposed framework. The modeling performance in estimation of PERCLOS provides robust and repeatable results in tests with manual and automated driving modes by an average RMSE of 0.117 (at a PERCLOS range of 0 to 1) and average High Probability Density percentage of 62.5%. We further hypothesized that there are biomarkers that encode the PERCLOS across different driving tests and participants. Using this solution, we identified possible biomarkers such as Theta and Delta powers. Results show that about 73% and 66% of the Theta and Delta powers which are selected as biomarkers are increasing as PERCLOS grows during the driving test. We argue that the proposed method is a robust and reliable solution to estimate drowsiness in real-time which opens the door in utilizing EEG-based measures in driver drowsiness detection systems.
Collapse
Affiliation(s)
- Sadegh Arefnezhad
- Institute of Automotive Engineering, Graz University of Technology, 8010, Graz, Austria.
| | - James Hamet
- Neurable Company, Boston, MA, 02108, USA.,Vistim Labs Company, Salt Lake City, UT, 84103, USA
| | - Arno Eichberger
- Institute of Automotive Engineering, Graz University of Technology, 8010, Graz, Austria
| | | | - Anja Ischebeck
- Institute of Psychology, University of Graz, 8010, Graz, Austria
| | - Ioana Victoria Koglbauer
- Institute of Engineering and Business Informatics, Graz University of Technology, Graz, 8010, Austria
| | - Maximilian Moser
- Human Research Institute, Weiz, 8160, Austria.,Chair of Department of Physiology, Medical University of Graz, 8036, Graz, Austria
| | - Ali Yousefi
- Neurable Company, Boston, MA, 02108, USA.,Department of Computer Science Worcester Polytechnic Institute, 100 Institute Road, MA, 01609, Worcester, USA
| |
Collapse
|
11
|
Reddy Katthi J, Ganapathy S. Deep Correlation Analysis for Audio-EEG Decoding. IEEE Trans Neural Syst Rehabil Eng 2021; 29:2742-2753. [PMID: 34874861 DOI: 10.1109/tnsre.2021.3129790] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
The electroencephalography (EEG), which is one of the easiest modes of recording brain activations in a non-invasive manner, is often distorted due to recording artifacts which adversely impacts the stimulus-response analysis. The most prominent techniques thus far attempt to improve the stimulus-response correlations using linear methods. In this paper, we propose a neural network based correlation analysis framework that significantly improves over the linear methods for auditory stimuli. A deep model is proposed for intra-subject audio-EEG analysis based on directly optimizing the correlation loss. Further, a neural network model with a shared encoder architecture is proposed for improving the inter-subject stimulus response correlations. These models attempt to suppress the EEG artifacts while preserving the components related to the stimulus. Several experiments are performed using EEG recordings from subjects listening to speech and music stimuli. In these experiments, we show that the deep models improve the Pearson correlation significantly over the linear methods (average absolute improvements of 7.4% in speech tasks and 29.3% in music tasks). We also analyze the impact of several model parameters on the stimulus-response correlation.
Collapse
|
12
|
Ki JJ, Dmochowski JP, Touryan J, Parra LC. Neural responses to natural visual motion are spatially selective across the visual field, with selectivity differing across brain areas and task. Eur J Neurosci 2021; 54:7609-7625. [PMID: 34679237 PMCID: PMC9298375 DOI: 10.1111/ejn.15503] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2021] [Revised: 09/16/2021] [Accepted: 10/07/2021] [Indexed: 11/28/2022]
Abstract
It is well established that neural responses to visual stimuli are enhanced at select locations in the visual field. Although spatial selectivity and the effects of spatial attention are well understood for discrete tasks (e.g. visual cueing), little is known for naturalistic experience that involves continuous dynamic visual stimuli (e.g. driving). Here, we assess the strength of neural responses across the visual space during a kart‐race game. Given the varying relevance of visual location in this task, we hypothesized that the strength of neural responses to movement will vary across the visual field, and it would differ between active play and passive viewing. To test this, we measure the correlation strength of scalp‐evoked potentials with optical flow magnitude at individual locations on the screen. We find that neural responses are strongly correlated at task‐relevant locations in visual space, extending beyond the focus of overt attention. Although the driver's gaze is directed upon the heading direction at the centre of the screen, neural responses were robust at the peripheral areas (e.g. roads and surrounding buildings). Importantly, neural responses to visual movement are broadly distributed across the scalp, with visual spatial selectivity differing across electrode locations. Moreover, during active gameplay, neural responses are enhanced at select locations in the visual space. Conventionally, spatial selectivity of neural response has been interpreted as an attentional gain mechanism. In the present study, the data suggest that different brain areas focus attention on different portions of the visual field that are task‐relevant, beyond the focus of overt attention.
Collapse
Affiliation(s)
- Jason J Ki
- Department of Biomedical Engineering, City College of the City University of New York, New York, New York, USA
| | - Jacek P Dmochowski
- Department of Biomedical Engineering, City College of the City University of New York, New York, New York, USA
| | | | - Lucas C Parra
- Department of Biomedical Engineering, City College of the City University of New York, New York, New York, USA
| |
Collapse
|
13
|
Lunner T, Alickovic E, Graversen C, Ng EHN, Wendt D, Keidser G. Three New Outcome Measures That Tap Into Cognitive Processes Required for Real-Life Communication. Ear Hear 2021; 41 Suppl 1:39S-47S. [PMID: 33105258 PMCID: PMC7676869 DOI: 10.1097/aud.0000000000000941] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2020] [Accepted: 07/11/2020] [Indexed: 11/29/2022]
Abstract
To increase the ecological validity of outcomes from laboratory evaluations of hearing and hearing devices, it is desirable to introduce more realistic outcome measures in the laboratory. This article presents and discusses three outcome measures that have been designed to go beyond traditional speech-in-noise measures to better reflect realistic everyday challenges. The outcome measures reviewed are: the Sentence-final Word Identification and Recall (SWIR) test that measures working memory performance while listening to speech in noise at ceiling performance; a neural tracking method that produces a quantitative measure of selective speech attention in noise; and pupillometry that measures changes in pupil dilation to assess listening effort while listening to speech in noise. According to evaluation data, the SWIR test provides a sensitive measure in situations where speech perception performance might be unaffected. Similarly, pupil dilation has also shown sensitivity in situations where traditional speech-in-noise measures are insensitive. Changes in working memory capacity and effort mobilization were found at positive signal-to-noise ratios (SNR), that is, at SNRs that might reflect everyday situations. Using stimulus reconstruction, it has been demonstrated that neural tracking is a robust method at determining to what degree a listener is attending to a specific talker in a typical cocktail party situation. Using both established and commercially available noise reduction schemes, data have further shown that all three measures are sensitive to variation in SNR. In summary, the new outcome measures seem suitable for testing hearing and hearing devices under more realistic and demanding everyday conditions than traditional speech-in-noise tests.
Collapse
Affiliation(s)
- Thomas Lunner
- Eriksholm Research Centre, Oticon A/S, Snekkersten, Denmark
- Department of Behavioural Sciences and Learning, Linnaeus Centre HEAD, Linköping University, Linköping, Sweden
- Department of Electrical Engineering, Division Automatic Control, Linköping University, Linköping, Sweden
- Department of Health Technology, Hearing Systems, Technical University of Denmark, Lyngby, Denmark
| | - Emina Alickovic
- Eriksholm Research Centre, Oticon A/S, Snekkersten, Denmark
- Department of Electrical Engineering, Division Automatic Control, Linköping University, Linköping, Sweden
| | | | - Elaine Hoi Ning Ng
- Department of Behavioural Sciences and Learning, Linnaeus Centre HEAD, Linköping University, Linköping, Sweden
- Oticon A/S, Kongebakken, Denmark
| | - Dorothea Wendt
- Eriksholm Research Centre, Oticon A/S, Snekkersten, Denmark
- Department of Health Technology, Hearing Systems, Technical University of Denmark, Lyngby, Denmark
| | - Gitte Keidser
- Eriksholm Research Centre, Oticon A/S, Snekkersten, Denmark
- Department of Behavioural Sciences and Learning, Linnaeus Centre HEAD, Linköping University, Linköping, Sweden
| |
Collapse
|
14
|
Rosenkranz M, Holtze B, Jaeger M, Debener S. EEG-Based Intersubject Correlations Reflect Selective Attention in a Competing Speaker Scenario. Front Neurosci 2021; 15:685774. [PMID: 34194296 PMCID: PMC8236636 DOI: 10.3389/fnins.2021.685774] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2021] [Accepted: 05/18/2021] [Indexed: 11/13/2022] Open
Abstract
Several solutions have been proposed to study the relationship between ongoing brain activity and natural sensory stimuli, such as running speech. Computing the intersubject correlation (ISC) has been proposed as one possible approach. Previous evidence suggests that ISCs between the participants' electroencephalogram (EEG) may be modulated by attention. The current study addressed this question in a competing-speaker paradigm, where participants (N = 41) had to attend to one of two concurrently presented speech streams. ISCs between participants' EEG were higher for participants attending to the same story compared to participants attending to different stories. Furthermore, we found that ISCs between individual and group data predicted whether an individual attended to the left or right speech stream. Interestingly, the magnitude of the shared neural response with others attending to the same story was related to the individual neural representation of the attended and ignored speech envelope. Overall, our findings indicate that ISC differences reflect the magnitude of selective attentional engagement to speech.
Collapse
Affiliation(s)
- Marc Rosenkranz
- Neuropsychology Lab, Department of Psychology, University of Oldenburg, Oldenburg, Germany
| | - Björn Holtze
- Neuropsychology Lab, Department of Psychology, University of Oldenburg, Oldenburg, Germany
| | - Manuela Jaeger
- Neuropsychology Lab, Department of Psychology, University of Oldenburg, Oldenburg, Germany.,Division Hearing, Fraunhofer Institute for Digital Media Technology IDMT, Speech and Audio Technology, Oldenburg, Germany
| | - Stefan Debener
- Neuropsychology Lab, Department of Psychology, University of Oldenburg, Oldenburg, Germany.,Cluster of Excellence Hearing4all, University of Oldenburg, Oldenburg, Germany.,Research Center for Neurosensory Science, University of Oldenburg, Oldenburg, Germany
| |
Collapse
|
15
|
de Cheveigné A, Slaney M, Fuglsang SA, Hjortkjaer J. Auditory stimulus-response modeling with a match-mismatch task. J Neural Eng 2021; 18. [PMID: 33849003 DOI: 10.1088/1741-2552/abf771] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2020] [Accepted: 04/13/2021] [Indexed: 11/12/2022]
Abstract
Objective.An auditory stimulus can be related to the brain response that it evokes by a stimulus-response model fit to the data. This offers insight into perceptual processes within the brain and is also of potential use for devices such as brain computer interfaces (BCIs). The quality of the model can be quantified by measuring the fit with a regression problem, or by applying it to a classification task and measuring its performance.Approach.Here we focus on amatch-mismatch(MM) task that entails deciding whether a segment of brain signal matches, via a model, the auditory stimulus that evoked it.Main results. Using these metrics, we describe a range of models of increasing complexity that we compare to methods in the literature, showing state-of-the-art performance. We document in detail one particular implementation, calibrated on a publicly-available database, that can serve as a robust reference to evaluate future developments.Significance.The MM task allows stimulus-response models to be evaluated in the limit of very high model accuracy, making it an attractive alternative to the more commonly used task of auditory attention detection. The MM task does not require class labels, so it is immune to mislabeling, and it is applicable to data recorded in listening scenarios with only one sound source, thus it is cheap to obtain large quantities of training and testing data. Performance metrics from this task, associated with regression accuracy, provide complementary insights into the relation between stimulus and response, as well as information about discriminatory power directly applicable to BCI applications.
Collapse
Affiliation(s)
- Alain de Cheveigné
- Laboratoire des Systèmes Perceptifs, Paris, CNRS UMR 8248, France.,Département d'Etudes Cognitives, Ecole Normale Supérieure, Paris, PSL, France.,UCL Ear Institute, London, United Kingdom.,Audition, DEC, ENS, 29 rue d'Ulm, 75230 Paris, France
| | - Malcolm Slaney
- Google Research, Machine Hearing Group, Mountain View, CA, United States of America
| | - Søren A Fuglsang
- Danish Research Centre for Magnetic Resonance, Centre for Functional and Diagnostic Imaging and Research, Copenhagen University Hospital Hvidovre, Copenhagen, Denmark
| | - Jens Hjortkjaer
- Hearing Systems Section, Department of Health Technology, Technical University of Denmark, Kgs. Lyngby, Denmark.,Danish Research Centre for Magnetic Resonance, Centre for Functional and Diagnostic Imaging and Research, Copenhagen University Hospital Hvidovre, Copenhagen, Denmark
| |
Collapse
|
16
|
Koide-Majima N, Majima K. Quantum-inspired canonical correlation analysis for exponentially large dimensional data. Neural Netw 2020; 135:55-67. [PMID: 33348241 DOI: 10.1016/j.neunet.2020.11.019] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2019] [Revised: 09/29/2020] [Accepted: 11/30/2020] [Indexed: 10/22/2022]
Abstract
Canonical correlation analysis (CCA) serves to identify statistical dependencies between pairs of multivariate data. However, its application to high-dimensional data is limited due to considerable computational complexity. As an alternative to the conventional CCA approach that requires polynomial computational time, we propose an algorithm that approximates CCA using quantum-inspired computations with computational time proportional to the logarithm of the input dimensionality. The computational efficiency and performance of the proposed quantum-inspired CCA (qiCCA) algorithm are experimentally evaluated on synthetic and real datasets. Furthermore, the fast computation provided by qiCCA allows directly applying CCA even after nonlinearly mapping raw input data into high-dimensional spaces. The conducted experiments demonstrate that, as a result of mapping raw input data into the high-dimensional spaces with the use of second-order monomials, qiCCA extracts more correlations compared with the linear CCA and achieves comparable performance with state-of-the-art nonlinear variants of CCA on several datasets. These results confirm the appropriateness of the proposed qiCCA and the high potential of quantum-inspired computations in analyzing high-dimensional data.
Collapse
Affiliation(s)
- Naoko Koide-Majima
- AI Science Research and Development Promotion Center, National Institute of Information and Communications Technology, Osaka 565-0871, Japan; Graduate School of Frontier Biosciences, Osaka University, Osaka 565-0871, Japan
| | - Kei Majima
- Graduate School of Informatics, Kyoto University, Kyoto 606-8501, Japan; Institute for Quantum Life Science, National Institutes for Quantum and Radiological Science and Technology, Chiba, 263-8555, Japan.
| |
Collapse
|
17
|
Laforge G, Gonzalez-Lara LE, Owen AM, Stojanoski B. Individualized assessment of residual cognition in patients with disorders of consciousness. Neuroimage Clin 2020; 28:102472. [PMID: 33395966 PMCID: PMC7652775 DOI: 10.1016/j.nicl.2020.102472] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2020] [Revised: 10/01/2020] [Accepted: 10/13/2020] [Indexed: 12/02/2022]
Abstract
Patients diagnosed with disorders of consciousness show minimal or inconsistent behavioural evidence of conscious awareness. However, using functional neuroimaging, recent research in clinical neuroscience has identified a subpopulation of these patients who reliably produce neural markers indicative of awareness. In this study, we recorded electroencephalograms during a response-free movie task to assess narrative processing in patients with disorders of consciousness. Thirteen patients diagnosed with a disorder of consciousness and 28 healthy controls participated in this study. We designed a movie-watching/listening paradigm involving two suspenseful movie clips, one audiovisual and one audio-only, and used electroencephalography to extract patterns of brain activity that were maximally correlated between subjects. These activity patterns served as electrophysiological indices of narrative processing, which were compared to the neural responses of patients during the same movies. Our analysis revealed two patterns of neural activity, one for each movie condition, that were significantly and reliably correlated between healthy participants. Of the twelve patients who watched the audiovisual movie, 25% produced a pattern of activity that was significantly correlated with the healthy group, while of the ten who listened to the audio narrative, 30% produced electrophysiological patterns similar to controls (one patient responded appropriately to both). The method presented here allows for rapid bedside assessment of higher-order cognitive processing in patients with disorders of consciousness. By leveraging the common neural response to movie stimuli, we were able to identify comparable patterns of brain activity in individual, behaviourally non-responsive patients, reflecting a capacity for narrative processing.
Collapse
Affiliation(s)
- Geoffrey Laforge
- The Brain and Mind Institute, The University of Western Ontario, London ON N6A 5B7, Canada.
| | - Laura E Gonzalez-Lara
- The Brain and Mind Institute, The University of Western Ontario, London ON N6A 5B7, Canada
| | - Adrian M Owen
- The Brain and Mind Institute, The University of Western Ontario, London ON N6A 5B7, Canada; The Department of Psychology, The University of Western Ontario, London ON N6A 5B7, Canada; The Department of Physiology and Pharmacology, The University of Western Ontario, London ON N6A 5B7, Canada
| | - Bobby Stojanoski
- The Brain and Mind Institute, The University of Western Ontario, London ON N6A 5B7, Canada; The Department of Psychology, The University of Western Ontario, London ON N6A 5B7, Canada
| |
Collapse
|
18
|
Alickovic E, Lunner T, Wendt D, Fiedler L, Hietkamp R, Ng EHN, Graversen C. Neural Representation Enhanced for Speech and Reduced for Background Noise With a Hearing Aid Noise Reduction Scheme During a Selective Attention Task. Front Neurosci 2020; 14:846. [PMID: 33071722 PMCID: PMC7533612 DOI: 10.3389/fnins.2020.00846] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2020] [Accepted: 07/20/2020] [Indexed: 12/23/2022] Open
Abstract
Objectives Selectively attending to a target talker while ignoring multiple interferers (competing talkers and background noise) is more difficult for hearing-impaired (HI) individuals compared to normal-hearing (NH) listeners. Such tasks also become more difficult as background noise levels increase. To overcome these difficulties, hearing aids (HAs) offer noise reduction (NR) schemes. The objective of this study was to investigate the effect of NR processing (inactive, where the NR feature was switched off, vs. active, where the NR feature was switched on) on the neural representation of speech envelopes across two different background noise levels [+3 dB signal-to-noise ratio (SNR) and +8 dB SNR] by using a stimulus reconstruction (SR) method. Design To explore how NR processing supports the listeners’ selective auditory attention, we recruited 22 HI participants fitted with HAs. To investigate the interplay between NR schemes, background noise, and neural representation of the speech envelopes, we used electroencephalography (EEG). The participants were instructed to listen to a target talker in front while ignoring a competing talker in front in the presence of multi-talker background babble noise. Results The results show that the neural representation of the attended speech envelope was enhanced by the active NR scheme for both background noise levels. The neural representation of the attended speech envelope at lower (+3 dB) SNR was shifted, approximately by 5 dB, toward the higher (+8 dB) SNR when the NR scheme was turned on. The neural representation of the ignored speech envelope was modulated by the NR scheme and was mostly enhanced in the conditions with more background noise. The neural representation of the background noise was modulated (i.e., reduced) by the NR scheme and was significantly reduced in the conditions with more background noise. The neural representation of the net sum of the ignored acoustic scene (ignored talker and background babble) was not modulated by the NR scheme but was significantly reduced in the conditions with a reduced level of background noise. Taken together, we showed that the active NR scheme enhanced the neural representation of both the attended and the ignored speakers and reduced the neural representation of background noise, while the net sum of the ignored acoustic scene was not enhanced. Conclusion Altogether our results support the hypothesis that the NR schemes in HAs serve to enhance the neural representation of speech and reduce the neural representation of background noise during a selective attention task. We contend that these results provide a neural index that could be useful for assessing the effects of HAs on auditory and cognitive processing in HI populations.
Collapse
Affiliation(s)
- Emina Alickovic
- Eriksholm Research Centre, Oticon A/S, Snekkersten, Denmark.,Department of Electrical Engineering, Linkoping University, Linköping, Sweden
| | - Thomas Lunner
- Eriksholm Research Centre, Oticon A/S, Snekkersten, Denmark.,Department of Electrical Engineering, Linkoping University, Linköping, Sweden.,Department of Health Technology, Technical University of Denmark, Lyngby, Denmark.,Department of Behavioral Sciences and Learning, Linkoping University, Linköping, Sweden
| | - Dorothea Wendt
- Eriksholm Research Centre, Oticon A/S, Snekkersten, Denmark.,Department of Health Technology, Technical University of Denmark, Lyngby, Denmark
| | - Lorenz Fiedler
- Eriksholm Research Centre, Oticon A/S, Snekkersten, Denmark
| | | | - Elaine Hoi Ning Ng
- Department of Behavioral Sciences and Learning, Linkoping University, Linköping, Sweden.,Oticon A/S, Smørum, Denmark
| | | |
Collapse
|
19
|
Zhuang X, Yang Z, Cordes D. A technical review of canonical correlation analysis for neuroscience applications. Hum Brain Mapp 2020; 41:3807-3833. [PMID: 32592530 PMCID: PMC7416047 DOI: 10.1002/hbm.25090] [Citation(s) in RCA: 70] [Impact Index Per Article: 17.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2020] [Accepted: 05/23/2020] [Indexed: 12/11/2022] Open
Abstract
Collecting comprehensive data sets of the same subject has become a standard in neuroscience research and uncovering multivariate relationships among collected data sets have gained significant attentions in recent years. Canonical correlation analysis (CCA) is one of the powerful multivariate tools to jointly investigate relationships among multiple data sets, which can uncover disease or environmental effects in various modalities simultaneously and characterize changes during development, aging, and disease progressions comprehensively. In the past 10 years, despite an increasing number of studies have utilized CCA in multivariate analysis, simple conventional CCA dominates these applications. Multiple CCA-variant techniques have been proposed to improve the model performance; however, the complicated multivariate formulations and not well-known capabilities have delayed their wide applications. Therefore, in this study, a comprehensive review of CCA and its variant techniques is provided. Detailed technical formulation with analytical and numerical solutions, current applications in neuroscience research, and advantages and limitations of each CCA-related technique are discussed. Finally, a general guideline in how to select the most appropriate CCA-related technique based on the properties of available data sets and particularly targeted neuroscience questions is provided.
Collapse
Affiliation(s)
- Xiaowei Zhuang
- Cleveland Clinic Lou Ruvo Center for Brain HealthLas VegasNevadaUSA
| | - Zhengshi Yang
- Cleveland Clinic Lou Ruvo Center for Brain HealthLas VegasNevadaUSA
| | - Dietmar Cordes
- Cleveland Clinic Lou Ruvo Center for Brain HealthLas VegasNevadaUSA
- University of ColoradoBoulderColoradoUSA
- Department of Brain HealthUniversity of NevadaLas VegasNevadaUSA
| |
Collapse
|
20
|
Ki JJ, Parra LC, Dmochowski JP. Visually evoked responses are enhanced when engaging in a video game. Eur J Neurosci 2020; 52:4695-4708. [PMID: 32735746 PMCID: PMC7818444 DOI: 10.1111/ejn.14924] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2020] [Revised: 06/26/2020] [Accepted: 07/15/2020] [Indexed: 12/02/2022]
Abstract
While it is well known that vision guides movement, less appreciated is that the motor cortex also provides input to the visual system. Here, we asked whether neural processing of visual stimuli is acutely modulated during motor activity, hypothesizing that visual evoked responses are enhanced when engaged in a motor task that depends on the visual stimulus. To test this, we told participants that their brain activity was controlling a video game that was in fact the playback of a prerecorded game. The deception, which was effective in half of participants, aimed to engage the motor system while avoiding evoked responses related to actual movement or somatosensation. In other trials, subjects actively played the game with keyboard control or passively watched a playback. The strength of visually evoked responses was measured as the temporal correlation between the continuous stimulus and the evoked potentials on the scalp. We found reduced correlation during passive viewing, but no difference between active and sham play. Alpha‐band (8–12 Hz) activity was reduced over central electrodes during sham play, indicating recruitment of motor cortex despite the absence of overt movement. To account for the potential increase of attention during gameplay, we conducted a second study with subjects counting screen items during viewing. We again found increased correlation during sham play, but no difference between counting and passive viewing. While we cannot fully rule out the involvement of attention, our findings do demonstrate an enhancement of visual evoked responses during active vision.
Collapse
Affiliation(s)
- Jason J Ki
- Department of Biomedical Engineering, City College of New York, New York, NY, USA
| | - Lucas C Parra
- Department of Biomedical Engineering, City College of New York, New York, NY, USA
| | - Jacek P Dmochowski
- Department of Biomedical Engineering, City College of New York, New York, NY, USA
| |
Collapse
|
21
|
Tozzi A, Ahmad MZ, Peters JF. Neural computing in four spatial dimensions. Cogn Neurodyn 2020; 15:349-357. [PMID: 33854648 DOI: 10.1007/s11571-020-09598-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2019] [Revised: 04/26/2020] [Accepted: 05/11/2020] [Indexed: 12/22/2022] Open
Abstract
Relationships among near set theory, shape maps and recent accounts of the Quantum Hall effect pave the way to neural networks computations performed in higher dimensions. We illustrate the operational procedure to build a real or artificial neural network able to detect, assess and quantify a fourth spatial dimension. We show how, starting from two-dimensional shapes embedded in a 2D topological charge pump, it is feasible to achieve the corresponding four-dimensional shapes, which encompass a larger amount of information. Synthesis of surface shape components, viewed topologically as shape descriptions in the form of feature vectors that vary over time, leads to a 4D view of cerebral activity. This novel, relatively straightforward architecture permits to increase the amount of available qbits in a fixed volume.
Collapse
Affiliation(s)
- Arturo Tozzi
- Center for Nonlinear Science, University of North Texas, 1155 Union Circle, #311427, Denton, TX 76203-5017 USA
| | - Muhammad Zubair Ahmad
- Department of Electrical and Computer Engineering, University of Manitoba, 75A Chancellor's Circle, Winnipeg, MB R3T 5V6 Canada
| | - James F Peters
- Department of Electrical and Computer Engineering, University of Manitoba, 75A Chancellor's Circle, Winnipeg, MB R3T 5V6 Canada
| |
Collapse
|
22
|
Kaneshiro B, Nguyen DT, Norcia AM, Dmochowski JP, Berger J. Natural music evokes correlated EEG responses reflecting temporal structure and beat. Neuroimage 2020; 214:116559. [PMID: 31978543 DOI: 10.1016/j.neuroimage.2020.116559] [Citation(s) in RCA: 29] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2019] [Revised: 12/23/2019] [Accepted: 01/14/2020] [Indexed: 11/17/2022] Open
Abstract
The brain activity of multiple subjects has been shown to synchronize during salient moments of natural stimuli, suggesting that correlation of neural responses indexes a brain state operationally termed 'engagement'. While past electroencephalography (EEG) studies have considered both auditory and visual stimuli, the extent to which these results generalize to music-a temporally structured stimulus for which the brain has evolved specialized circuitry-is less understood. Here we investigated neural correlation during natural music listening by recording EEG responses from N=48 adult listeners as they heard real-world musical works, some of which were temporally disrupted through shuffling of short-term segments (measures), reversal, or randomization of phase spectra. We measured correlation between multiple neural responses (inter-subject correlation) and between neural responses and stimulus envelope fluctuations (stimulus-response correlation) in the time and frequency domains. Stimuli retaining basic musical features, such as rhythm and melody, elicited significantly higher behavioral ratings and neural correlation than did phase-scrambled controls. However, while unedited songs were self-reported as most pleasant, time-domain correlations were highest during measure-shuffled versions. Frequency-domain measures of correlation (coherence) peaked at frequencies related to the musical beat, although the magnitudes of these spectral peaks did not explain the observed temporal correlations. Our findings show that natural music evokes significant inter-subject and stimulus-response correlations, and suggest that the neural correlates of musical 'engagement' may be distinct from those of enjoyment.
Collapse
Affiliation(s)
- Blair Kaneshiro
- Center for Computer Research in Music and Acoustics, Stanford University, Stanford, CA, USA; Center for the Study of Language and Information, Stanford University, Stanford, CA, USA; Department of Otolaryngology Head & Neck Surgery, Stanford University School of Medicine, Palo Alto, CA, USA.
| | - Duc T Nguyen
- Center for Computer Research in Music and Acoustics, Stanford University, Stanford, CA, USA; Center for the Study of Language and Information, Stanford University, Stanford, CA, USA; Department of Biomedical Engineering, City College of New York, New York, NY, USA
| | - Anthony M Norcia
- Department of Psychology, Stanford University, Stanford, CA, USA
| | - Jacek P Dmochowski
- Department of Biomedical Engineering, City College of New York, New York, NY, USA; Department of Psychology, Stanford University, Stanford, CA, USA
| | - Jonathan Berger
- Center for Computer Research in Music and Acoustics, Stanford University, Stanford, CA, USA
| |
Collapse
|
23
|
Imhof MA, Schmälzle R, Renner B, Schupp HT. Strong health messages increase audience brain coupling. Neuroimage 2020; 216:116527. [PMID: 31954843 DOI: 10.1016/j.neuroimage.2020.116527] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2019] [Revised: 12/03/2019] [Accepted: 01/07/2020] [Indexed: 10/25/2022] Open
Abstract
Mass media messaging is central for health communication. The success of these efforts, however, depends on whether health messages resonate with their target audiences. Here, we used electroencephalography (EEG) to capture brain responses of young adults - an important target group for alcohol prevention - while they viewed real-life video messages of varying perceived message effectiveness about risky alcohol use. We found that strong messages, which were rated to be more effective, prompted enhanced inter-subject correlation (ISC). In further analyses, we linked ISC to subsequent drinking behavior change and used time-resolved EEG-ISC to model functional neuroimaging data (fMRI) of an independent audience. The EEG measure was not only related to sensory-perceptual brain regions, but also to regions previously related to successful messaging, i.e., cortical midline regions and the insula. The findings suggest EEG-ISC as a marker for audience engagement and effectiveness of naturalistic health messages, which could quantify the impact of mass communication within the brains of small target audiences.
Collapse
Affiliation(s)
- Martin A Imhof
- Department of Psychology, University of Konstanz, 78457, Konstanz, Germany; Centre for the Advanced Study of Collective Behaviour, University of Konstanz, 78457, Konstanz, Germany.
| | - Ralf Schmälzle
- Department of Communication, Michigan State University, East Lansing, MI, 48824, USA
| | - Britta Renner
- Department of Psychology, University of Konstanz, 78457, Konstanz, Germany; Centre for the Advanced Study of Collective Behaviour, University of Konstanz, 78457, Konstanz, Germany
| | - Harald T Schupp
- Department of Psychology, University of Konstanz, 78457, Konstanz, Germany; Centre for the Advanced Study of Collective Behaviour, University of Konstanz, 78457, Konstanz, Germany
| |
Collapse
|
24
|
Das N, Vanthornhout J, Francart T, Bertrand A. Stimulus-aware spatial filtering for single-trial neural response and temporal response function estimation in high-density EEG with applications in auditory research. Neuroimage 2020; 204:116211. [PMID: 31546052 PMCID: PMC7355237 DOI: 10.1016/j.neuroimage.2019.116211] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2019] [Revised: 08/30/2019] [Accepted: 09/17/2019] [Indexed: 12/21/2022] Open
Abstract
A common problem in neural recordings is the low signal-to-noise ratio (SNR), particularly when using non-invasive techniques like magneto- or electroencephalography (M/EEG). To address this problem, experimental designs often include repeated trials, which are then averaged to improve the SNR or to infer statistics that can be used in the design of a denoising spatial filter. However, collecting enough repeated trials is often impractical and even impossible in some paradigms, while analyses on existing data sets may be hampered when these do not contain such repeated trials. Therefore, we present a data-driven method that takes advantage of the knowledge of the presented stimulus, to achieve a joint noise reduction and dimensionality reduction without the need for repeated trials. The method first estimates the stimulus-driven neural response using the given stimulus, which is then used to find a set of spatial filters that maximize the SNR based on a generalized eigenvalue decomposition. As the method is fully data-driven, the dimensionality reduction enables researchers to perform their analyses without having to rely on their knowledge of brain regions of interest, which increases accuracy and reduces the human factor in the results. In the context of neural tracking of a speech stimulus using EEG, our method resulted in more accurate short-term temporal response function (TRF) estimates, higher correlations between predicted and actual neural responses, and higher attention decoding accuracies compared to existing TRF-based decoding methods. We also provide an extensive discussion on the central role played by the generalized eigenvalue decomposition in various denoising methods in the literature, and address the conceptual similarities and differences with our proposed method.
Collapse
Affiliation(s)
- Neetha Das
- Dept. Electrical Engineering (ESAT), Stadius Center for Dynamical Systems, Signal Processing and Data Analytics, KU Leuven, Kasteelpark Arenberg 10, B-3001, Leuven, Belgium; Dept. Neurosciences, ExpORL, KU Leuven, Herestraat 49 Bus 721, B-3000, Leuven, Belgium.
| | - Jonas Vanthornhout
- Dept. Neurosciences, ExpORL, KU Leuven, Herestraat 49 Bus 721, B-3000, Leuven, Belgium
| | - Tom Francart
- Dept. Neurosciences, ExpORL, KU Leuven, Herestraat 49 Bus 721, B-3000, Leuven, Belgium
| | - Alexander Bertrand
- Dept. Electrical Engineering (ESAT), Stadius Center for Dynamical Systems, Signal Processing and Data Analytics, KU Leuven, Kasteelpark Arenberg 10, B-3001, Leuven, Belgium.
| |
Collapse
|
25
|
Transfer learning of deep neural network representations for fMRI decoding. J Neurosci Methods 2019; 328:108319. [DOI: 10.1016/j.jneumeth.2019.108319] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2019] [Revised: 06/06/2019] [Accepted: 06/17/2019] [Indexed: 11/22/2022]
|
26
|
Lesenfants D, Vanthornhout J, Verschueren E, Francart T. Data-driven spatial filtering for improved measurement of cortical tracking of multiple representations of speech. J Neural Eng 2019; 16:066017. [PMID: 31426053 DOI: 10.1088/1741-2552/ab3c92] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
OBJECTIVE Measurement of the cortical tracking of continuous speech from electroencephalography (EEG) recordings using a forward model is an important tool in auditory neuroscience. Usually the stimulus is represented by its temporal envelope. Recently, the phonetic representation of speech was successfully introduced in English. We aim to show that the EEG prediction from phoneme-related speech features is possible in Dutch. The method requires a manual channel selection based on visual inspection or prior knowledge to obtain a summary measure of cortical tracking. We evaluate a method to (1) remove non-stimulus-related activity from the EEG signals to be predicted, and (2) automatically select the channels of interest. APPROACH Eighteen participants listened to a Flemish story, while their EEG was recorded. Subject-specific and grand-average temporal response functions were determined between the EEG activity in different frequency bands and several stimulus features: the envelope, spectrogram, phonemes, phonetic features or a combination. The temporal response functions were used to predict EEG from the stimulus, and the predicted was compared with the recorded EEG, yielding a measure of cortical tracking of stimulus features. A spatial filter was calculated based on the generalized eigenvalue decomposition (GEVD), and the effect on EEG prediction accuracy was determined. MAIN RESULTS A model including both low- and high-level speech representations was able to better predict the brain responses to the speech than a model only including low-level features. The inclusion of a GEVD-based spatial filter in the model increased the prediction accuracy of cortical responses to each speech feature at both single-subject (270% improvement) and group-level (310%). SIGNIFICANCE We showed that the inclusion of acoustical and phonetic speech information and the addition of a data-driven spatial filter allow improved modelling of the relationship between the speech and its brain responses and offer an automatic channel selection.
Collapse
Affiliation(s)
- D Lesenfants
- Department of Neurosciences, Experimental Oto-Rhino-Laryngology, KU Leuven, Belgium
| | | | | | | |
Collapse
|
27
|
Activity of the inferior parietal cortex is modulated by visual feedback delay in the robot hand illusion. Sci Rep 2019; 9:10030. [PMID: 31296914 PMCID: PMC6624271 DOI: 10.1038/s41598-019-46527-8] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2018] [Accepted: 06/29/2019] [Indexed: 12/19/2022] Open
Abstract
The robot hand illusion (RoHI) is the perception of self-ownership and self-agency of a virtual (robot) hand that moves consistently with one's own. The phenomenon shows that self-attribution can be established via temporal integration of visual and movement information. Our previous study showed that participants felt significantly greater RoHI (sense of self-ownership and sense of self-agency) when visuomotor temporal discrepancies were less than 200 ms. A weaker RoHI effect (sense of self-agency only) was observed when temporal discrepancies were between 300 and 500 ms. Here, we used functional near-infrared spectroscopy (fNIRS) to investigate brain activity associated with the RoHI under different visual feedback delays (100 ms, 400 ms, 700 ms). We found that the angular and supramarginal gyri exhibited significant activation in the 100-ms feedback condition. ANOVA indicated a significant difference between the 100-ms condition and the other conditions (p < 0.01). These results demonstrate that activity in the posterior parietal cortex was modulated by the delay between the motor command and the visual feedback of the virtual hand movements. Thus, we propose that the inferior parietal cortex is essential for integrating motor and visual information to distinguish one's own body from others.
Collapse
|
28
|
Alickovic E, Lunner T, Gustafsson F, Ljung L. A Tutorial on Auditory Attention Identification Methods. Front Neurosci 2019; 13:153. [PMID: 30941002 PMCID: PMC6434370 DOI: 10.3389/fnins.2019.00153] [Citation(s) in RCA: 29] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2018] [Accepted: 02/11/2019] [Indexed: 01/14/2023] Open
Abstract
Auditory attention identification methods attempt to identify the sound source of a listener's interest by analyzing measurements of electrophysiological data. We present a tutorial on the numerous techniques that have been developed in recent decades, and we present an overview of current trends in multivariate correlation-based and model-based learning frameworks. The focus is on the use of linear relations between electrophysiological and audio data. The way in which these relations are computed differs. For example, canonical correlation analysis (CCA) finds a linear subset of electrophysiological data that best correlates to audio data and a similar subset of audio data that best correlates to electrophysiological data. Model-based (encoding and decoding) approaches focus on either of these two sets. We investigate the similarities and differences between these linear model philosophies. We focus on (1) correlation-based approaches (CCA), (2) encoding/decoding models based on dense estimation, and (3) (adaptive) encoding/decoding models based on sparse estimation. The specific focus is on sparsity-driven adaptive encoding models and comparing the methodology in state-of-the-art models found in the auditory literature. Furthermore, we outline the main signal processing pipeline for how to identify the attended sound source in a cocktail party environment from the raw electrophysiological data with all the necessary steps, complemented with the necessary MATLAB code and the relevant references for each step. Our main aim is to compare the methodology of the available methods, and provide numerical illustrations to some of them to get a feeling for their potential. A thorough performance comparison is outside the scope of this tutorial.
Collapse
Affiliation(s)
- Emina Alickovic
- Department of Electrical Engineering, Linkoping University, Linkoping, Sweden
- Eriksholm Research Centre, Oticon A/S, Snekkersten, Denmark
| | - Thomas Lunner
- Department of Electrical Engineering, Linkoping University, Linkoping, Sweden
- Eriksholm Research Centre, Oticon A/S, Snekkersten, Denmark
- Hearing Systems, Department of Health Technology, Technical University of Denmark, Lyngby, Denmark
- Swedish Institute for Disability Research, Linnaeus Centre HEAD, Linkoping University, Linkoping, Sweden
| | - Fredrik Gustafsson
- Department of Electrical Engineering, Linkoping University, Linkoping, Sweden
| | - Lennart Ljung
- Department of Electrical Engineering, Linkoping University, Linkoping, Sweden
| |
Collapse
|
29
|
Abstract
OBJECTIVE Speech signals have a remarkable ability to entrain brain activity to the rapid fluctuations of speech sounds. For instance, one can readily measure a correlation of the sound amplitude with the evoked responses of the electroencephalogram (EEG), and the strength of this correlation is indicative of whether the listener is attending to the speech. In this study we asked whether this stimulus-response correlation is also predictive of speech intelligibility. APPROACH We hypothesized that when a listener fails to understand the speech in adverse hearing conditions, attention wanes and stimulus-response correlation also drops. To test this, we measure a listener's ability to detect words in noisy speech while recording their brain activity using EEG. We alter intelligibility without changing the acoustic stimulus by pairing it with congruent and incongruent visual speech. MAIN RESULTS For almost all subjects we found that an improvement in speech detection coincided with an increase in correlation between the noisy speech and the EEG measured over a period of 30 min. SIGNIFICANCE We conclude that simultaneous recordings of the perceived sound and the corresponding EEG response may be a practical tool to assess speech intelligibility in the context of hearing aids.
Collapse
Affiliation(s)
- Ivan Iotzov
- Biomedical Engineering, City College of New York, New York City, NY, United States of America
| | | |
Collapse
|
30
|
Haufe S, DeGuzman P, Henin S, Arcaro M, Honey CJ, Hasson U, Parra LC. Elucidating relations between fMRI, ECoG, and EEG through a common natural stimulus. Neuroimage 2018; 179:79-91. [PMID: 29902585 PMCID: PMC6063527 DOI: 10.1016/j.neuroimage.2018.06.016] [Citation(s) in RCA: 40] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2018] [Revised: 05/21/2018] [Accepted: 06/05/2018] [Indexed: 12/31/2022] Open
Abstract
Human brain mapping relies heavily on fMRI, ECoG and EEG, which capture different physiological signals. Relationships between these signals have been established in the context of specific tasks or during resting state, often using spatially confined concurrent recordings in animals. But it is not certain whether these correlations generalize to other contexts relevant for human cognitive neuroscience. Here, we address the case of complex naturalistic stimuli and ask two basic questions. First, how reliable are the responses evoked by a naturalistic audio-visual stimulus in each of these imaging methods, and second, how similar are stimulus-related responses across methods? To this end, we investigated a wide range of brain regions and frequency bands. We presented the same movie clip twice to three different cohorts of subjects (NEEG = 45, NfMRI = 11, NECoG = 5) and assessed stimulus-driven correlations across viewings and between imaging methods, thereby ruling out task-irrelevant confounds. All three imaging methods had similar repeat-reliability across viewings when fMRI and EEG data were averaged across subjects, highlighting the potential to achieve large signal-to-noise ratio by leveraging large sample sizes. The fMRI signal correlated positively with high-frequency ECoG power across multiple task-related cortical structures but positively with low-frequency EEG and ECoG power. In contrast to previous studies, these correlations were as strong for low-frequency as for high frequency ECoG. We also observed links between fMRI and infra-slow EEG voltage fluctuations. These results extend previous findings to the case of natural stimulus processing.
Collapse
Affiliation(s)
- Stefan Haufe
- Technische Universität Berlin, Berlin, Germany; City College New York, New York, NY, USA; Columbia University, New York, NY, USA.
| | | | - Simon Henin
- NYU Langone Medical Center, New York, NY, USA
| | | | | | - Uri Hasson
- Princeton University, Princeton, NJ, USA
| | - Lucas C Parra
- City College New York, New York, NY, USA; Neuromatters LLC, New York, NY, USA.
| |
Collapse
|
31
|
|
32
|
Delis I, Dmochowski JP, Sajda P, Wang Q. Correlation of neural activity with behavioral kinematics reveals distinct sensory encoding and evidence accumulation processes during active tactile sensing. Neuroimage 2018; 175:12-21. [PMID: 29580968 PMCID: PMC5960621 DOI: 10.1016/j.neuroimage.2018.03.035] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2017] [Revised: 02/21/2018] [Accepted: 03/17/2018] [Indexed: 12/16/2022] Open
Abstract
Many real-world decisions rely on active sensing, a dynamic process for directing our sensors (e.g. eyes or fingers) across a stimulus to maximize information gain. Though ecologically pervasive, limited work has focused on identifying neural correlates of the active sensing process. In tactile perception, we often make decisions about an object/surface by actively exploring its shape/texture. Here we investigate the neural correlates of active tactile decision-making by simultaneously measuring electroencephalography (EEG) and finger kinematics while subjects interrogated a haptic surface to make perceptual judgments. Since sensorimotor behavior underlies decision formation in active sensing tasks, we hypothesized that the neural correlates of decision-related processes would be detectable by relating active sensing to neural activity. Novel brain-behavior correlation analysis revealed that three distinct EEG components, localizing to right-lateralized occipital cortex (LOC), middle frontal gyrus (MFG), and supplementary motor area (SMA), respectively, were coupled with active sensing as their activity significantly correlated with finger kinematics. To probe the functional role of these components, we fit their single-trial-couplings to decision-making performance using a hierarchical-drift-diffusion-model (HDDM), revealing that the LOC modulated the encoding of the tactile stimulus whereas the MFG predicted the rate of information integration towards a choice. Interestingly, the MFG disappeared from components uncovered from control subjects performing active sensing but not required to make perceptual decisions. By uncovering the neural correlates of distinct stimulus encoding and evidence accumulation processes, this study delineated, for the first time, the functional role of cortical areas in active tactile decision-making.
Collapse
Affiliation(s)
- Ioannis Delis
- Department of Biomedical Engineering, Columbia University, New York, NY, 10027, USA
| | - Jacek P Dmochowski
- Department of Biomedical Engineering, City College of New York, New York, NY, 10031, USA
| | - Paul Sajda
- Department of Biomedical Engineering, Columbia University, New York, NY, 10027, USA; Data Science Institute, Columbia University, New York, NY, 10027, USA.
| | - Qi Wang
- Department of Biomedical Engineering, Columbia University, New York, NY, 10027, USA.
| |
Collapse
|