1
|
Mudrik L, Hirschhorn R, Korisky U. Taking consciousness for real: Increasing the ecological validity of the study of conscious vs. unconscious processes. Neuron 2024; 112:1642-1656. [PMID: 38653247 PMCID: PMC11100345 DOI: 10.1016/j.neuron.2024.03.031] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2024] [Revised: 03/23/2024] [Accepted: 03/29/2024] [Indexed: 04/25/2024]
Abstract
The study of consciousness has developed well-controlled, rigorous methods for manipulating and measuring consciousness. Yet, in the process, experimental paradigms grew farther away from everyday conscious and unconscious processes, which raises the concern of ecological validity. In this review, we suggest that the field can benefit from adopting a more ecological approach, akin to other fields of cognitive science. There, this approach challenged some existing hypotheses, yielded stronger effects, and enabled new research questions. We argue that such a move is critical for studying consciousness, where experimental paradigms tend to be artificial and small effect sizes are relatively prevalent. We identify three paths for doing so-changing the stimuli and experimental settings, changing the measures, and changing the research questions themselves-and review works that have already started implementing such approaches. While acknowledging the inherent challenges, we call for increasing ecological validity in consciousness studies.
Collapse
Affiliation(s)
- Liad Mudrik
- School of Psychological Sciences, Tel Aviv University, Tel Aviv, Israel; Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel.
| | - Rony Hirschhorn
- Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel
| | - Uri Korisky
- School of Psychological Sciences, Tel Aviv University, Tel Aviv, Israel
| |
Collapse
|
2
|
Yang B, Intoy J, Rucci M. Eye blinks as a visual processing stage. Proc Natl Acad Sci U S A 2024; 121:e2310291121. [PMID: 38564641 PMCID: PMC11009678 DOI: 10.1073/pnas.2310291121] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2023] [Accepted: 02/12/2024] [Indexed: 04/04/2024] Open
Abstract
Humans blink their eyes frequently during normal viewing, more often than it seems necessary for keeping the cornea well lubricated. Since the closure of the eyelid disrupts the image on the retina, eye blinks are commonly assumed to be detrimental to visual processing. However, blinks also provide luminance transients rich in spatial information to neural pathways highly sensitive to temporal changes. Here, we report that the luminance modulations from blinks enhance visual sensitivity. By coupling high-resolution eye tracking in human observers with modeling of blink transients and spectral analysis of visual input signals, we show that blinking increases the power of retinal stimulation and that this effect significantly enhances visibility despite the time lost in exposure to the external scene. We further show that, as predicted from the spectral content of input signals, this enhancement is selective for stimuli at low spatial frequencies and occurs irrespective of whether the luminance transients are actively generated or passively experienced. These findings indicate that, like eye movements, blinking acts as a computational component of a visual processing strategy that uses motor behavior to reformat spatial information into the temporal domain.
Collapse
Affiliation(s)
- Bin Yang
- Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY14627
- Center for Visual Science, University of Rochester, Rochester, NY14627
| | - Janis Intoy
- Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY14627
- Center for Visual Science, University of Rochester, Rochester, NY14627
| | - Michele Rucci
- Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY14627
- Center for Visual Science, University of Rochester, Rochester, NY14627
| |
Collapse
|
3
|
Marciano D, Staveland BR, Lin JJ, Saez I, Hsu M, Knight RT. Electrophysiological signatures of inequity-dependent reward encoding in the human OFC. Cell Rep 2023; 42:112865. [PMID: 37494185 DOI: 10.1016/j.celrep.2023.112865] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2022] [Revised: 06/12/2023] [Accepted: 07/10/2023] [Indexed: 07/28/2023] Open
Abstract
Social decision making requires the integration of reward valuation and social cognition systems, both dependent on the orbitofrontal cortex (OFC). How these two OFC functions interact is largely unknown. We recorded intracranial activity from the OFC of ten patients making choices in a social context where reward inequity with a social counterpart varied and could be either advantageous or disadvantageous. We find that OFC high-frequency activity (HFA; 70-150 Hz) encodes self-reward, consistent with previous reports. We also observe encoding of the social counterpart's reward, as well as the type of inequity being experienced. Additionally, we find evidence of inequity-dependent reward encoding: depending on the type of inequity, electrodes rapidly and reversibly switch between different reward-encoding profiles. These results provide direct evidence for encoding of self- and other rewards in the human OFC and highlight the dynamic nature of encoding in the OFC as a function of social context.
Collapse
Affiliation(s)
- Deborah Marciano
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA 94720, USA; Haas School of Business, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Brooke R Staveland
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Jack J Lin
- Department of Neurology, University of California, Davis, Davis, CA 95616, USA; Center for Mind and Brain, University of California, Davis, Davis, CA 95616, USA
| | - Ignacio Saez
- Departments of Neuroscience, Neurosurgery and Neurology, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA.
| | - Ming Hsu
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA 94720, USA; Haas School of Business, University of California, Berkeley, Berkeley, CA 94720, USA.
| | - Robert T Knight
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA 94720, USA; Department of Psychology, University of California, Berkeley, Berkeley, CA 94720, USA.
| |
Collapse
|
4
|
Callara AL, Greco A, Scilingo EP, Bonfiglio L. Neuronal correlates of eyeblinks are an expression of primary consciousness phenomena. Sci Rep 2023; 13:12617. [PMID: 37537328 PMCID: PMC10400571 DOI: 10.1038/s41598-023-39500-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2022] [Accepted: 07/26/2023] [Indexed: 08/05/2023] Open
Abstract
The blinking rate far exceeds that required for moistening the cornea and changes depending on whether a person is resting or engaged in cognitive tasks. During ecological cognitive tasks (such as speaking, reading, and watching videos), blinks occur at breakpoints of attention suggesting a role in information segmentation, but the close relationship between cognition dynamics and blink timing still escapes a full understanding. The aim of the present study is to seek (1) if there is a temporal relationship between blink events and the consecutive steps of cognitive processing, and (2) if blink timing and the intensity of blink-related EEG responses are affected by task-relevance of stimuli. Our results show that, in a classical visual oddball task, (i) the occurrence of blinks is influenced by stimuli, irrespective of their relevance, (ii) blinks following relevant stimuli are only apparently delayed due to the need of finalizing a behavioural response, and (iii) stimulus relevance does not affect the intensity of the blink-related EEG response. This evidence reinforce the idea that blinks are not emitted until the last step of the processing sequence has been completed and suggests that blink-related EEG responses are generated by primary consciousness phenomena which are considered by their nature non-modulable (all-or-nothing) phenomena.
Collapse
Affiliation(s)
- Alejandro Luis Callara
- Dipartimento di Ingegneria dell'Informazione, University of Pisa, Via G. Caruso 16, 56122, Pisa, Italy
- Research Center "E. Piaggio", University of Pisa, Largo Lucio Lazzarino 1, 56122, Pisa, Italy
| | - Alberto Greco
- Dipartimento di Ingegneria dell'Informazione, University of Pisa, Via G. Caruso 16, 56122, Pisa, Italy
- Research Center "E. Piaggio", University of Pisa, Largo Lucio Lazzarino 1, 56122, Pisa, Italy
| | - Enzo Pasquale Scilingo
- Dipartimento di Ingegneria dell'Informazione, University of Pisa, Via G. Caruso 16, 56122, Pisa, Italy
- Research Center "E. Piaggio", University of Pisa, Largo Lucio Lazzarino 1, 56122, Pisa, Italy
| | - Luca Bonfiglio
- Department of Translational Research and New Technologies in Medicine and Surgery, University of Pisa, Pisa, Italy.
- Unit of Developmental Neurorehabilitation, Maternal and Child Department, Pisa University Hospital, Pisa, Italy.
| |
Collapse
|
5
|
Broday-Dvir R, Norman Y, Harel M, Mehta AD, Malach R. Perceptual stability reflected in neuronal pattern similarities in human visual cortex. Cell Rep 2023; 42:112614. [PMID: 37285270 DOI: 10.1016/j.celrep.2023.112614] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2022] [Revised: 03/14/2023] [Accepted: 05/22/2023] [Indexed: 06/09/2023] Open
Abstract
The magnitude of neuronal activation is commonly considered a critical factor for conscious perception of visual content. However, this dogma contrasts with the phenomenon of rapid adaptation, in which the magnitude of neuronal activation drops dramatically in a rapid manner while the visual stimulus and the conscious experience it elicits remain stable. Here, we report that the profiles of multi-site activation patterns and their relational geometry-i.e., the similarity distances between activation patterns, as revealed using intracranial electroencephalographic (iEEG) recordings-are sustained during extended visual stimulation despite the major magnitude decrease. These results are compatible with the hypothesis that conscious perceptual content is associated with the neuronal pattern profiles and their similarity distances, rather than the overall activation magnitude, in human visual cortex.
Collapse
Affiliation(s)
- Rotem Broday-Dvir
- Department of Brain Sciences, Weizmann Institute of Science, Rehovot 76100, Israel
| | - Yitzhak Norman
- Department of Neurological Surgery, University of California, San Francisco, San Francisco, CA 94143, USA
| | - Michal Harel
- Department of Brain Sciences, Weizmann Institute of Science, Rehovot 76100, Israel
| | - Ashesh D Mehta
- Department of Neurosurgery, Zucker School of Medicine at Hofstra/Northwell, and the Feinstein Institute for Medical Research, Manhasset, NY 11030, USA
| | - Rafael Malach
- Department of Brain Sciences, Weizmann Institute of Science, Rehovot 76100, Israel.
| |
Collapse
|
6
|
Turner KL, Gheres KW, Drew PJ. Relating Pupil Diameter and Blinking to Cortical Activity and Hemodynamics across Arousal States. J Neurosci 2023; 43:949-964. [PMID: 36517240 PMCID: PMC9908322 DOI: 10.1523/jneurosci.1244-22.2022] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2022] [Revised: 12/06/2022] [Accepted: 12/09/2022] [Indexed: 12/23/2022] Open
Abstract
Arousal state affects neural activity and vascular dynamics in the cortex, with sleep associated with large changes in the local field potential and increases in cortical blood flow. We investigated the relationship between pupil diameter and blink rate with neural activity and blood volume in the somatosensory cortex in male and female unanesthetized, head-fixed mice. We monitored these variables while the mice were awake, during periods of rapid eye movement (REM), and non-rapid eye movement (NREM) sleep. Pupil diameter was smaller during sleep than in the awake state. Changes in pupil diameter were coherent with both gamma-band power and blood volume in the somatosensory cortex, but the strength and sign of this relationship varied with arousal state. We observed a strong negative correlation between pupil diameter and both gamma-band power and blood volume during periods of awake rest and NREM sleep, although the correlations between pupil diameter and these signals became positive during periods of alertness, active whisking, and REM. Blinking was associated with increases in arousal and decreases in blood volume when the mouse was asleep. Bilateral coherence in gamma-band power and in blood volume dropped following awake blinking, indicating a reset of neural and vascular activity. Using only eye metrics (pupil diameter and eye motion), we could determine the arousal state of the mouse ('Awake,' 'NREM,' 'REM') with >90% accuracy with a 5 s resolution. There is a strong relationship between pupil diameter and hemodynamics signals in mice, reflecting the pronounced effects of arousal on cerebrovascular dynamics.SIGNIFICANCE STATEMENT Determining arousal state is a critical component of any neuroscience experiment. Pupil diameter and blinking are influenced by arousal state, as are hemodynamics signals in the cortex. We investigated the relationship between cortical hemodynamics and pupil diameter and found that pupil diameter was strongly related to the blood volume in the cortex. Mice were more likely to be awake after blinking than before, and blinking resets neural activity. Pupil diameter and eye motion can be used as a reliable, noninvasive indicator of arousal state. As mice transition from wake to sleep and back again over a timescale of seconds, monitoring pupil diameter and eye motion permits the noninvasive detection of sleep events during behavioral or resting-state experiments.
Collapse
Affiliation(s)
- Kevin L Turner
- Department of Biomedical Engineering, Pennsylvania State University, University Park, Pennsylvania 16802
- Center for Neural Engineering, Pennsylvania State University, University Park, Pennsylvania 16802
| | - Kyle W Gheres
- Center for Neural Engineering, Pennsylvania State University, University Park, Pennsylvania 16802
- Departments of Engineering Science and Mechanics
| | - Patrick J Drew
- Department of Biomedical Engineering, Pennsylvania State University, University Park, Pennsylvania 16802
- Center for Neural Engineering, Pennsylvania State University, University Park, Pennsylvania 16802
- Departments of Engineering Science and Mechanics
- Biology and Neurosurgery, Pennsylvania State University, University Park, Pennsylvania 16802
| |
Collapse
|
7
|
Demiral ŞB, Kure Liu C, Benveniste H, Tomasi D, Volkow ND. Activation of brain arousal networks coincident with eye blinks during resting state. Cereb Cortex 2023:6991186. [PMID: 36653022 DOI: 10.1093/cercor/bhad001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2022] [Revised: 12/27/2022] [Accepted: 12/28/2022] [Indexed: 01/20/2023] Open
Abstract
Eye-blinking has been implicated in arousal and attention. Here we test the hypothesis that blinking-moments represent arousal surges associated with activation of the ascending arousal network (AAN) and its thalamic projections. For this purpose, we explored the temporal relationship between eye-blinks and fMRI BOLD activity in AAN and thalamic nuclei, as well as whole brain cluster corrected activations during eyes-open, resting-state fMRI scanning. We show that BOLD activations in the AAN nuclei peaked prior to the eye blinks and in thalamic nuclei peaked prior to and during the blink, consistent with the role of eye blinking in arousal surges. Additionally, we showed visual cortex peak activation prior to the eye blinks, providing further evidence of the visual cortex's role in arousal, and document cerebellar peak activation post eye blinks, which might reflect downstream engagement from arousal surges.
Collapse
Affiliation(s)
- Şükrü Barış Demiral
- Laboratory of Neuroimaging, National Institute on Alcohol Abuse and Alcoholism, National Institutes of Health, Bethesda 20892, MD, USA
| | - Christopher Kure Liu
- Laboratory of Neuroimaging, National Institute on Alcohol Abuse and Alcoholism, National Institutes of Health, Bethesda 20892, MD, USA
| | - Helene Benveniste
- Department of Anesthesiology, Yale University, New Haven, CT 06510, USA
| | - Dardo Tomasi
- Laboratory of Neuroimaging, National Institute on Alcohol Abuse and Alcoholism, National Institutes of Health, Bethesda 20892, MD, USA
| | - Nora D Volkow
- Laboratory of Neuroimaging, National Institute on Alcohol Abuse and Alcoholism, National Institutes of Health, Bethesda 20892, MD, USA.,National Institute on Drug Abuse, National Institutes of Health, Bethesda, MD 20892, USA
| |
Collapse
|
8
|
Mercier MR, Dubarry AS, Tadel F, Avanzini P, Axmacher N, Cellier D, Vecchio MD, Hamilton LS, Hermes D, Kahana MJ, Knight RT, Llorens A, Megevand P, Melloni L, Miller KJ, Piai V, Puce A, Ramsey NF, Schwiedrzik CM, Smith SE, Stolk A, Swann NC, Vansteensel MJ, Voytek B, Wang L, Lachaux JP, Oostenveld R. Advances in human intracranial electroencephalography research, guidelines and good practices. Neuroimage 2022; 260:119438. [PMID: 35792291 DOI: 10.1016/j.neuroimage.2022.119438] [Citation(s) in RCA: 47] [Impact Index Per Article: 23.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2021] [Revised: 05/23/2022] [Accepted: 06/30/2022] [Indexed: 12/11/2022] Open
Abstract
Since the second-half of the twentieth century, intracranial electroencephalography (iEEG), including both electrocorticography (ECoG) and stereo-electroencephalography (sEEG), has provided an intimate view into the human brain. At the interface between fundamental research and the clinic, iEEG provides both high temporal resolution and high spatial specificity but comes with constraints, such as the individual's tailored sparsity of electrode sampling. Over the years, researchers in neuroscience developed their practices to make the most of the iEEG approach. Here we offer a critical review of iEEG research practices in a didactic framework for newcomers, as well addressing issues encountered by proficient researchers. The scope is threefold: (i) review common practices in iEEG research, (ii) suggest potential guidelines for working with iEEG data and answer frequently asked questions based on the most widespread practices, and (iii) based on current neurophysiological knowledge and methodologies, pave the way to good practice standards in iEEG research. The organization of this paper follows the steps of iEEG data processing. The first section contextualizes iEEG data collection. The second section focuses on localization of intracranial electrodes. The third section highlights the main pre-processing steps. The fourth section presents iEEG signal analysis methods. The fifth section discusses statistical approaches. The sixth section draws some unique perspectives on iEEG research. Finally, to ensure a consistent nomenclature throughout the manuscript and to align with other guidelines, e.g., Brain Imaging Data Structure (BIDS) and the OHBM Committee on Best Practices in Data Analysis and Sharing (COBIDAS), we provide a glossary to disambiguate terms related to iEEG research.
Collapse
|
9
|
Sklar AY, Kardosh R, Hassin RR. From non-conscious processing to conscious events: a minimalist approach. Neurosci Conscious 2021; 2021:niab026. [PMID: 34676105 PMCID: PMC8524171 DOI: 10.1093/nc/niab026] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2021] [Revised: 06/23/2021] [Accepted: 10/11/2021] [Indexed: 01/22/2023] Open
Abstract
The minimalist approach that we develop here is a framework that allows to appreciate how non-conscious processing and conscious contents shape human cognition, broadly defined. It is composed of three simple principles. First, cognitive processes are inherently non-conscious, while their inputs and (interim) outputs may be consciously experienced. Second, non-conscious processes and elements of the cognitive architecture prioritize information for conscious experiences. Third, conscious events are composed of series of conscious contents and non-conscious processes, with increased duration leading to more opportunity for processing. The narrowness of conscious experiences is conceptualized here as a solution to the problem of channeling the plethora of non-conscious processes into action and communication processes that are largely serial. The framework highlights the importance of prioritization for consciousness, and we provide an illustrative review of three main factors that shape prioritization-stimulus strength, motivational relevance and mental accessibility. We further discuss when and how this framework (i) is compatible with previous theories, (ii) enables new understandings of established findings and models, and (iii) generates new predictions and understandings.
Collapse
Affiliation(s)
- Asael Y Sklar
- Edmond & Lily Safra Center for Brain Sciences, The Hebrew University Edmond J. Safra Campus, Jerusalem 9190401, Israel
| | - Rasha Kardosh
- Psychology Department, The Hebrew University Mount Scopus, Jerusalem 91905, Israel
| | - Ran R Hassin
- James Marshall Chair of Psychology, Psychology Department & The Federmann Center for the Study of Rationality, The Hebrew University Mount Scopus, Jerusalem 91905, Israel
| |
Collapse
|
10
|
Resolving visual motion through perceptual gaps. Trends Cogn Sci 2021; 25:978-991. [PMID: 34489180 DOI: 10.1016/j.tics.2021.07.017] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2021] [Revised: 07/27/2021] [Accepted: 07/30/2021] [Indexed: 01/22/2023]
Abstract
Perceptual gaps can be caused by objects in the foreground temporarily occluding objects in the background or by eyeblinks, which briefly but frequently interrupt visual information. Resolving visual motion across perceptual gaps is particularly challenging, as object position changes during the gap. We examine how visual motion is maintained and updated through externally driven (occlusion) and internally driven (eyeblinks) perceptual gaps. Focusing on both phenomenology and potential mechanisms such as suppression, extrapolation, and integration, we present a framework for how perceptual gaps are resolved over space and time. We finish by highlighting critical questions and directions for future work.
Collapse
|
11
|
Malach R. Local neuronal relational structures underlying the contents of human conscious experience. Neurosci Conscious 2021; 2021:niab028. [PMID: 34513028 PMCID: PMC8415036 DOI: 10.1093/nc/niab028] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2021] [Revised: 08/02/2021] [Accepted: 08/12/2021] [Indexed: 01/04/2023] Open
Abstract
While most theories of consciousness posit some kind of dependence on global network activities, I consider here an alternative, localist perspective-in which localized cortical regions each underlie the emergence of a unique category of conscious experience. Under this perspective, the large-scale activation often found in the cortex is a consequence of the complexity of typical conscious experiences rather than an obligatory condition for the emergence of conscious awareness-which can flexibly shift, depending on the richness of its contents, from local to more global activation patterns. This perspective fits a massive body of human imaging, recordings, lesions and stimulation data but opens a fundamental problem: how can the information, defining each content, be derived locally in each cortical region. Here, I will discuss a solution echoing pioneering structuralist ideas in which the content of a conscious experience is defined by its relationship to all other contents within an experiential category. In neuronal terms, this relationship structure between contents is embodied by the local geometry of similarity distances between cortical activation patterns generated during each conscious experience, likely mediated via networks of local neuronal connections. Thus, in order for any conscious experience to appear in an individual's mind, two central conditions must be met. First, a specific configural pattern ("bar-code") of neuronal activity must appear within a local relational geometry, i.e. a cortical area. Second, the individual neurons underlying the activated pattern must be bound into a unified functional ensemble through a burst of recurrent neuronal firing: local "ignitions".
Collapse
Affiliation(s)
- Rafael Malach
- Department of Brain Sciences, Weizmann Institute of Science, 200 Herzl St. POB 76100, Rehovot, Israel
- The School of Psychological Sciences, Tel Aviv University, P.O. Box 39040, Tel Aviv 6997801, Israel
| |
Collapse
|
12
|
Leszczynski M, Chaieb L, Staudigl T, Enkirch SJ, Fell J, Schroeder CE. Neural activity in the human anterior thalamus during natural vision. Sci Rep 2021; 11:17480. [PMID: 34471183 PMCID: PMC8410783 DOI: 10.1038/s41598-021-96588-x] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2020] [Accepted: 08/11/2021] [Indexed: 12/23/2022] Open
Abstract
In natural vision humans and other primates explore environment by active sensing, using saccadic eye movements to relocate the fovea and sample different bits of information multiple times per second. Saccades induce a phase reset of ongoing neuronal oscillations in primary and higher-order visual cortices and in the medial temporal lobe. As a result, neuron ensembles are shifted to a common state at the time visual input propagates through the system (i.e., just after fixation). The extent of the brain’s circuitry that is modulated by saccades is not yet known. Here, we evaluate the possibility that saccadic phase reset impacts the anterior nuclei of the thalamus (ANT). Using recordings in the human thalamus of three surgical patients during natural vision, we found that saccades and visual stimulus onset both modulate neural activity, but with distinct field potential morphologies. Specifically, we found that fixation-locked field potentials had a component that preceded saccade onset. It was followed by an early negativity around 50 ms after fixation onset which is significantly faster than any response to visual stimulus presentation. The timing of these events suggests that the ANT is predictively modulated before the saccadic eye movement. We also found oscillatory phase concentration, peaking at 3–4 Hz, coincident with suppression of Broadband High-frequency Activity (BHA; 80–180 Hz), both locked to fixation onset supporting the idea that neural oscillations in these nuclei are reorganized to a low excitability state right after fixation onset. These findings show that during real-world natural visual exploration neural dynamics in the human ANT is influenced by visual and oculomotor events, which supports the idea that ANT, apart from their contribution to episodic memory, also play a role in natural vision.
Collapse
Affiliation(s)
- Marcin Leszczynski
- Department of Psychiatry, College of Physicians and Surgeons, Columbia University Medical Center, 1051 Riverside Drive Kolb Annex Rm 561, New York, NY, 10032, USA. .,Translational Neuroscience Division, Center for Biomedical Imaging and Neuromodulation, Nathan Kline Institute, Orangeburg, NY, USA.
| | - Leila Chaieb
- Department of Epileptology, University Hospital Bonn, Bonn, Germany
| | - Tobias Staudigl
- Department of Psychology, Ludwig-Maximilians-Universität München, Munich, Germany
| | | | - Juergen Fell
- Department of Epileptology, University Hospital Bonn, Bonn, Germany
| | - Charles E Schroeder
- Department of Psychiatry, College of Physicians and Surgeons, Columbia University Medical Center, 1051 Riverside Drive Kolb Annex Rm 561, New York, NY, 10032, USA.,Translational Neuroscience Division, Center for Biomedical Imaging and Neuromodulation, Nathan Kline Institute, Orangeburg, NY, USA
| |
Collapse
|
13
|
Wienke C, Bartsch MV, Vogelgesang L, Reichert C, Hinrichs H, Heinze HJ, Dürschmid S. Mind-wandering Is Accompanied by Both Local Sleep and Enhanced Processes of Spatial Attention Allocation. Cereb Cortex Commun 2021; 2:tgab001. [PMID: 34296151 PMCID: PMC8153027 DOI: 10.1093/texcom/tgab001] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2020] [Revised: 12/28/2020] [Accepted: 12/30/2020] [Indexed: 11/30/2022] Open
Abstract
Mind-wandering (MW) is a subjective, cognitive phenomenon, in which thoughts move away from the task toward an internal train of thoughts, possibly during phases of neuronal sleep-like activity (local sleep, LS). MW decreases cortical processing of external stimuli and is assumed to decouple attention from the external world. Here, we directly tested how indicators of LS, cortical processing, and attentional selection change in a pop-out visual search task during phases of MW. Participants’ brain activity was recorded using magnetoencephalography, MW was assessed via self-report using randomly interspersed probes. As expected, the performance decreased under MW. Consistent with the occurrence of LS, MW was accompanied by a decrease in high-frequency activity (HFA, 80–150 Hz) and an increase in slow wave activity (SWA, 1–6 Hz). In contrast, visual attentional selection as indexed by the N2pc component was enhanced during MW with the N2pc amplitude being directly linked to participants’ performance. This observation clearly contradicts accounts of attentional decoupling that would predict a decrease in attention-related responses to external stimuli during MW. Together, our results suggest that MW occurs during phases of LS with processes of attentional target selection being upregulated, potentially to compensate for the mental distraction during MW.
Collapse
Affiliation(s)
- Christian Wienke
- Department of Neurology, Otto-von-Guericke University, Leipziger Str. 44, 39120 Magdeburg, Germany
| | - Mandy V Bartsch
- Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Brenneckestr. 6, 39118 Magdeburg, Germany
| | - Lena Vogelgesang
- Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Brenneckestr. 6, 39118 Magdeburg, Germany
| | - Christoph Reichert
- Forschungscampus STIMULATE, Otto-von-Guericke University, Universitätsplatz 2, 39106 Magdeburg, Germany.,Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Brenneckestr. 6, 39118 Magdeburg, Germany.,CBBS - center of behavioral brain sciences, Otto-von-Guericke University, Universitätsplatz 2, 39106 Magdeburg, Germany
| | - Hermann Hinrichs
- Department of Neurology, Otto-von-Guericke University, Leipziger Str. 44, 39120 Magdeburg, Germany.,Forschungscampus STIMULATE, Otto-von-Guericke University, Universitätsplatz 2, 39106 Magdeburg, Germany.,Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Brenneckestr. 6, 39118 Magdeburg, Germany.,CBBS - center of behavioral brain sciences, Otto-von-Guericke University, Universitätsplatz 2, 39106 Magdeburg, Germany.,German Center for Neurodegenerative Diseases (DZNE), Leipziger Str. 44, 39120 Magdeburg, Germany
| | - Hans-Jochen Heinze
- Department of Neurology, Otto-von-Guericke University, Leipziger Str. 44, 39120 Magdeburg, Germany.,Forschungscampus STIMULATE, Otto-von-Guericke University, Universitätsplatz 2, 39106 Magdeburg, Germany.,Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Brenneckestr. 6, 39118 Magdeburg, Germany.,CBBS - center of behavioral brain sciences, Otto-von-Guericke University, Universitätsplatz 2, 39106 Magdeburg, Germany.,German Center for Neurodegenerative Diseases (DZNE), Leipziger Str. 44, 39120 Magdeburg, Germany
| | - Stefan Dürschmid
- Department of Neurology, Otto-von-Guericke University, Leipziger Str. 44, 39120 Magdeburg, Germany.,Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Brenneckestr. 6, 39118 Magdeburg, Germany
| |
Collapse
|
14
|
Norman Y, Raccah O, Liu S, Parvizi J, Malach R. Hippocampal ripples and their coordinated dialogue with the default mode network during recent and remote recollection. Neuron 2021; 109:2767-2780.e5. [PMID: 34297916 DOI: 10.1016/j.neuron.2021.06.020] [Citation(s) in RCA: 36] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2021] [Revised: 05/13/2021] [Accepted: 06/16/2021] [Indexed: 12/19/2022]
Abstract
Hippocampal ripples are prominent synchronization events generated by hippocampal neuronal assemblies. To date, ripples have been primarily associated with navigational memory in rodents and short-term episodic recollections in humans. Here, we uncover different profiles of ripple activity in the human hippocampus during the retrieval of recent and remote autobiographical events and semantic facts. We found that the ripple rate increased significantly before reported recall compared to control conditions. Patterns of ripple activity across multiple hippocampal sites demonstrated remarkable specificity for memory type. Intriguingly, these ripple patterns revealed a semantization dimension, in which patterns associated with autobiographical contents become similar to those of semantic memory as a function of memory age. Finally, widely distributed sites across the neocortex exhibited ripple-coupled activations during recollection, with the strongest activation found within the default mode network. Our results thus reveal a key role for hippocampal ripples in orchestrating hippocampal-cortical communication across large-scale networks involved in conscious recollection.
Collapse
Affiliation(s)
- Yitzhak Norman
- Department of Neurobiology, Weizmann Institute of Science, Rehovot 76100, Israel.
| | - Omri Raccah
- Laboratory of Behavioral and Cognitive Neuroscience, Stanford Human Intracranial Cognitive Electrophysiology Program, Department of Neurology and Neurological Sciences, Stanford University, Stanford, CA 94305, USA
| | - Su Liu
- Laboratory of Behavioral and Cognitive Neuroscience, Stanford Human Intracranial Cognitive Electrophysiology Program, Department of Neurology and Neurological Sciences, Stanford University, Stanford, CA 94305, USA
| | - Josef Parvizi
- Laboratory of Behavioral and Cognitive Neuroscience, Stanford Human Intracranial Cognitive Electrophysiology Program, Department of Neurology and Neurological Sciences, Stanford University, Stanford, CA 94305, USA
| | - Rafael Malach
- Department of Neurobiology, Weizmann Institute of Science, Rehovot 76100, Israel.
| |
Collapse
|
15
|
Kern M, Schulze-Bonhage A, Ball T. Blink- and saccade-related suppression effects in early visual areas of the human brain: Intracranial EEG investigations during natural viewing conditions. Neuroimage 2021; 230:117788. [PMID: 33503480 DOI: 10.1016/j.neuroimage.2021.117788] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2020] [Revised: 01/07/2021] [Accepted: 01/12/2021] [Indexed: 01/07/2023] Open
Abstract
Blinks and saccades, both ubiquitous in natural viewing conditions, cause rapid changes of visual inputs that are hardly consciously perceived. The neural dynamics in early visual areas of the human brain underlying this remarkable visual stability are still incompletely understood. We used electrocorticography (ECoG) from electrodes directly implanted on the human early visual areas V1, V2, V3d/v, V4d/v and the fusiform gyrus to investigate blink- and saccade-related neuronal suppression effects during non-experimental, free viewing conditions. We found a characteristic, biphasic, broadband gamma power decrease-increase pattern in all investigated visual areas. During saccades, a decrease in gamma power clearly preceded eye movement onset, at least in V1. This may indicate that cortical information processing is actively suppressed in human early visual areas before and during saccades, which then possibly mediates perceptual visual suppression. The following eye movement offset-related increase in gamma power may indicate the recovery of visual perception and the resumption of visual processing.
Collapse
Affiliation(s)
- Markus Kern
- Neuromedical AI Lab, Department of Neurosurgery, Medical Center - University of Freiburg, Engelbergerstr.21, D-79106 Freiburg im Breisgau, Germany; Epilepsy Center, Department of Neurosurgery, Medical Center - University of Freiburg, Freiburg, Germany; Neurobiology and Biophysics, Faculty of Biology, University of Freiburg, Schänzlestrasse 1, 79104 Freiburg, Germany; BrainLinks-BrainTools Cluster of Excellence, University of Freiburg, Freiburg, Germany.
| | - Andreas Schulze-Bonhage
- Epilepsy Center, Department of Neurosurgery, Medical Center - University of Freiburg, Freiburg, Germany; BrainLinks-BrainTools Cluster of Excellence, University of Freiburg, Freiburg, Germany
| | - Tonio Ball
- Neuromedical AI Lab, Department of Neurosurgery, Medical Center - University of Freiburg, Engelbergerstr.21, D-79106 Freiburg im Breisgau, Germany; Epilepsy Center, Department of Neurosurgery, Medical Center - University of Freiburg, Freiburg, Germany; BrainLinks-BrainTools Cluster of Excellence, University of Freiburg, Freiburg, Germany
| |
Collapse
|
16
|
Pinzuti E, Wollstadt P, Gutknecht A, Tüscher O, Wibral M. Measuring spectrally-resolved information transfer. PLoS Comput Biol 2020; 16:e1008526. [PMID: 33370259 PMCID: PMC7793276 DOI: 10.1371/journal.pcbi.1008526] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2020] [Revised: 01/08/2021] [Accepted: 11/12/2020] [Indexed: 12/13/2022] Open
Abstract
Information transfer, measured by transfer entropy, is a key component of distributed computation. It is therefore important to understand the pattern of information transfer in order to unravel the distributed computational algorithms of a system. Since in many natural systems distributed computation is thought to rely on rhythmic processes a frequency resolved measure of information transfer is highly desirable. Here, we present a novel algorithm, and its efficient implementation, to identify separately frequencies sending and receiving information in a network. Our approach relies on the invertible maximum overlap discrete wavelet transform (MODWT) for the creation of surrogate data in the computation of transfer entropy and entirely avoids filtering of the original signals. The approach thereby avoids well-known problems due to phase shifts or the ineffectiveness of filtering in the information theoretic setting. We also show that measuring frequency-resolved information transfer is a partial information decomposition problem that cannot be fully resolved to date and discuss the implications of this issue. Last, we evaluate the performance of our algorithm on simulated data and apply it to human magnetoencephalography (MEG) recordings and to local field potential recordings in the ferret. In human MEG we demonstrate top-down information flow in temporal cortex from very high frequencies (above 100Hz) to both similarly high frequencies and to frequencies around 20Hz, i.e. a complex spectral configuration of cortical information transmission that has not been described before. In the ferret we show that the prefrontal cortex sends information at low frequencies (4-8 Hz) to early visual cortex (V1), while V1 receives the information at high frequencies (> 125 Hz).
Collapse
Affiliation(s)
| | - Patricia Wollstadt
- MEG Unit, Brain Imaging Center, Goethe University, Frankfurt/Main, Germany
| | - Aaron Gutknecht
- Campus Institute for Dynamics of Biological Networks, Georg August University, Göttingen, Germany
| | - Oliver Tüscher
- Leibniz Institute for Resilience Research, Mainz, Germany
- Department of Psychiatry and Psychotherapy, Johannes Gutenberg University of Mainz, Mainz, Germany
| | - Michael Wibral
- MEG Unit, Brain Imaging Center, Goethe University, Frankfurt/Main, Germany
- Campus Institute for Dynamics of Biological Networks, Georg August University, Göttingen, Germany
| |
Collapse
|
17
|
Thézé R, Giraud AL, Mégevand P. The phase of cortical oscillations determines the perceptual fate of visual cues in naturalistic audiovisual speech. SCIENCE ADVANCES 2020; 6:6/45/eabc6348. [PMID: 33148648 PMCID: PMC7673697 DOI: 10.1126/sciadv.abc6348] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/05/2020] [Accepted: 09/17/2020] [Indexed: 06/11/2023]
Abstract
When we see our interlocutor, our brain seamlessly extracts visual cues from their face and processes them along with the sound of their voice, making speech an intrinsically multimodal signal. Visual cues are especially important in noisy environments, when the auditory signal is less reliable. Neuronal oscillations might be involved in the cortical processing of audiovisual speech by selecting which sensory channel contributes more to perception. To test this, we designed computer-generated naturalistic audiovisual speech stimuli where one mismatched phoneme-viseme pair in a key word of sentences created bistable perception. Neurophysiological recordings (high-density scalp and intracranial electroencephalography) revealed that the precise phase angle of theta-band oscillations in posterior temporal and occipital cortex of the right hemisphere was crucial to select whether the auditory or the visual speech cue drove perception. We demonstrate that the phase of cortical oscillations acts as an instrument for sensory selection in audiovisual speech processing.
Collapse
Affiliation(s)
- Raphaël Thézé
- Department of Basic Neurosciences, Faculty of Medicine, University of Geneva, 1202 Geneva, Switzerland
| | - Anne-Lise Giraud
- Department of Basic Neurosciences, Faculty of Medicine, University of Geneva, 1202 Geneva, Switzerland
| | - Pierre Mégevand
- Department of Basic Neurosciences, Faculty of Medicine, University of Geneva, 1202 Geneva, Switzerland.
- Division of Neurology, Department of Clinical Neurosciences, Geneva University Hospitals, 1205 Geneva, Switzerland
| |
Collapse
|
18
|
Mégevand P, Mercier MR, Groppe DM, Zion Golumbic E, Mesgarani N, Beauchamp MS, Schroeder CE, Mehta AD. Crossmodal Phase Reset and Evoked Responses Provide Complementary Mechanisms for the Influence of Visual Speech in Auditory Cortex. J Neurosci 2020; 40:8530-8542. [PMID: 33023923 PMCID: PMC7605423 DOI: 10.1523/jneurosci.0555-20.2020] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2020] [Revised: 07/27/2020] [Accepted: 08/31/2020] [Indexed: 12/26/2022] Open
Abstract
Natural conversation is multisensory: when we can see the speaker's face, visual speech cues improve our comprehension. The neuronal mechanisms underlying this phenomenon remain unclear. The two main alternatives are visually mediated phase modulation of neuronal oscillations (excitability fluctuations) in auditory neurons and visual input-evoked responses in auditory neurons. Investigating this question using naturalistic audiovisual speech with intracranial recordings in humans of both sexes, we find evidence for both mechanisms. Remarkably, auditory cortical neurons track the temporal dynamics of purely visual speech using the phase of their slow oscillations and phase-related modulations in broadband high-frequency activity. Consistent with known perceptual enhancement effects, the visual phase reset amplifies the cortical representation of concomitant auditory speech. In contrast to this, and in line with earlier reports, visual input reduces the amplitude of evoked responses to concomitant auditory input. We interpret the combination of improved phase tracking and reduced response amplitude as evidence for more efficient and reliable stimulus processing in the presence of congruent auditory and visual speech inputs.SIGNIFICANCE STATEMENT Watching the speaker can facilitate our understanding of what is being said. The mechanisms responsible for this influence of visual cues on the processing of speech remain incompletely understood. We studied these mechanisms by recording the electrical activity of the human brain through electrodes implanted surgically inside the brain. We found that visual inputs can operate by directly activating auditory cortical areas, and also indirectly by modulating the strength of cortical responses to auditory input. Our results help to understand the mechanisms by which the brain merges auditory and visual speech into a unitary perception.
Collapse
Affiliation(s)
- Pierre Mégevand
- Department of Neurosurgery, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York 11549
- Feinstein Institutes for Medical Research, Manhasset, New York 11030
- Department of Basic Neurosciences, Faculty of Medicine, University of Geneva, 1211 Geneva, Switzerland
| | - Manuel R Mercier
- Department of Neurology, Montefiore Medical Center, Bronx, New York 10467
- Department of Neuroscience, Albert Einstein College of Medicine, Bronx, New York 10461
- Institut de Neurosciences des Systèmes, Aix Marseille University, INSERM, 13005 Marseille, France
| | - David M Groppe
- Department of Neurosurgery, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York 11549
- Feinstein Institutes for Medical Research, Manhasset, New York 11030
- The Krembil Neuroscience Centre, University Health Network, Toronto, Ontario M5T 1M8, Canada
| | - Elana Zion Golumbic
- The Gonda Brain Research Center, Bar Ilan University, Ramat Gan 5290002, Israel
| | - Nima Mesgarani
- Department of Electrical Engineering, Columbia University, New York, New York 10027
| | - Michael S Beauchamp
- Department of Neurosurgery, Baylor College of Medicine, Houston, Texas 77030
| | - Charles E Schroeder
- Nathan S. Kline Institute, Orangeburg, New York 10962
- Department of Psychiatry, Columbia University, New York, New York 10032
| | - Ashesh D Mehta
- Department of Neurosurgery, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York 11549
- Feinstein Institutes for Medical Research, Manhasset, New York 11030
| |
Collapse
|
19
|
Fiebelkorn IC, Kastner S. Spike Timing in the Attention Network Predicts Behavioral Outcome Prior to Target Selection. Neuron 2020; 109:177-188.e4. [PMID: 33098762 DOI: 10.1016/j.neuron.2020.09.039] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2020] [Revised: 07/08/2020] [Accepted: 09/25/2020] [Indexed: 10/23/2022]
Abstract
There has been little evidence linking changes in spiking activity that occur prior to a spatially predictable target (i.e., prior to target selection) to behavioral outcomes, despite such preparatory changes being widely assumed to enhance the sensitivity of sensory processing. We simultaneously recorded from frontal and parietal nodes of the attention network while macaques performed a spatial cueing task. When anticipating a spatially predictable target, different patterns of coupling between spike timing and the oscillatory phase in local field potentials-but not changes in spike rate-were predictive of different behavioral outcomes. These behaviorally relevant differences in local and between-region synchronization occurred among specific cell types that were defined based on their sensory and motor properties, providing insight into the mechanisms underlying enhanced sensory processing prior to target selection. We propose that these changes in neural synchronization reflect differential anticipatory engagement of the network nodes and functional units that shape attention-related sampling.
Collapse
Affiliation(s)
- Ian C Fiebelkorn
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA.
| | - Sabine Kastner
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA; Department of Psychology, Princeton University, Princeton, NJ 08544, USA
| |
Collapse
|
20
|
Strappini F, Wilf M, Karp O, Goldberg H, Harel M, Furman-Haran E, Golan T, Malach R. Resting-State Activity in High-Order Visual Areas as a Window into Natural Human Brain Activations. Cereb Cortex 2020; 29:3618-3635. [PMID: 30395164 DOI: 10.1093/cercor/bhy242] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2017] [Revised: 08/30/2018] [Accepted: 09/06/2018] [Indexed: 02/05/2023] Open
Abstract
A major limitation of conventional human brain research has been its basis in highly artificial laboratory experiments. Due to technical constraints, little is known about the nature of cortical activations during ecological real life. We have previously proposed the "spontaneous trait reactivation (STR)" hypothesis arguing that resting-state patterns, which emerge spontaneously in the absence of external stimulus, reflect the statistics of habitual cortical activations during real life. Therefore, these patterns can serve as a window into daily life cortical activity. A straightforward prediction of this hypothesis is that spontaneous patterns should preferentially correlate to patterns generated by naturalistic stimuli compared with artificial ones. Here we targeted high-level category-selective visual areas and tested this prediction by comparing BOLD functional connectivity patterns formed during rest to patterns formed in response to naturalistic stimuli, as well as to more artificial category-selective, dynamic stimuli. Our results revealed a significant correlation between the resting-state patterns and functional connectivity patterns generated by naturalistic stimuli. Furthermore, the correlations to naturalistic stimuli were significantly higher than those found between resting-state patterns and those generated by artificial control stimuli. These findings provide evidence of a stringent link between spontaneous patterns and the activation patterns during natural vision.
Collapse
Affiliation(s)
| | - Meytal Wilf
- Neurobiology Department, Weizmann Institute of Science, Rehovot, Israel.,Department of Clinical Neurosciences, MySpace Lab, Lausanne University Hospital, Lausanne, Switzerland
| | - Ofer Karp
- Neurobiology Department, Weizmann Institute of Science, Rehovot, Israel
| | - Hagar Goldberg
- Neurobiology Department, Weizmann Institute of Science, Rehovot, Israel
| | - Michal Harel
- Neurobiology Department, Weizmann Institute of Science, Rehovot, Israel
| | - Edna Furman-Haran
- Life Sciences Core Facilities Department, Weizmann Institute of Science, Rehovot, Israel
| | - Tal Golan
- The Edmund and Lily Safra Center for Brain Sciences, Hebrew University of Jerusalem, Jerusalem, Israel
| | - Rafael Malach
- Neurobiology Department, Weizmann Institute of Science, Rehovot, Israel
| |
Collapse
|
21
|
Abstract
We blink more often than required for maintaining the corneal tear film. Whether there are perceptual or cognitive consequences of blinks that may justify their high frequency is unclear. Previous findings showed that blinks may indicate switches between large-scale cortical networks, such as dorsal attention and default-mode networks. Thus, blinks may trigger a refresh of visual attention. Yet, this has so far not been confirmed behaviorally. Here, we tested the effect of blinks on visual performance in a series of rapid serial visual presentation tasks. In Experiment 1, participants had to identify a target digit embedded in a random stream of letter distractors, presented foveally for 60 ms each. Participants blinked once during the presentation stream. In a separate condition, blinks were simulated by shutter glasses. Detection performance was enhanced (up to 13% point increase in accuracy) for targets appearing up to 300 ms after eye blinks. Performance boosts were stronger for voluntary blinks than artificial blinks. This performance boost was also replicated with more naturalistic stimuli (Experiment 2). We conclude that eye blinks lead to attentional benefits for object recognition in the period after reopening of the eyelids and may be used strategically for temporarily boosting visual performance.
Collapse
Affiliation(s)
- Jit Wei A Ang
- Psychology Programme, School of Social Sciences, Nanyang Technological University, Singapore., https://orcid.org/0000-0001-8271-5031
| | - Gerrit W Maus
- Psychology Programme, School of Social Sciences, Nanyang Technological University, Singapore., https://orcid.org/0000-0003-4843-3748
| |
Collapse
|
22
|
Responses to Visual Speech in Human Posterior Superior Temporal Gyrus Examined with iEEG Deconvolution. J Neurosci 2020; 40:6938-6948. [PMID: 32727820 PMCID: PMC7470920 DOI: 10.1523/jneurosci.0279-20.2020] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2020] [Revised: 06/01/2020] [Accepted: 06/02/2020] [Indexed: 12/22/2022] Open
Abstract
Experimentalists studying multisensory integration compare neural responses to multisensory stimuli with responses to the component modalities presented in isolation. This procedure is problematic for multisensory speech perception since audiovisual speech and auditory-only speech are easily intelligible but visual-only speech is not. To overcome this confound, we developed intracranial encephalography (iEEG) deconvolution. Individual stimuli always contained both auditory and visual speech, but jittering the onset asynchrony between modalities allowed for the time course of the unisensory responses and the interaction between them to be independently estimated. We applied this procedure to electrodes implanted in human epilepsy patients (both male and female) over the posterior superior temporal gyrus (pSTG), a brain area known to be important for speech perception. iEEG deconvolution revealed sustained positive responses to visual-only speech and larger, phasic responses to auditory-only speech. Confirming results from scalp EEG, responses to audiovisual speech were weaker than responses to auditory-only speech, demonstrating a subadditive multisensory neural computation. Leveraging the spatial resolution of iEEG, we extended these results to show that subadditivity is most pronounced in more posterior aspects of the pSTG. Across electrodes, subadditivity correlated with visual responsiveness, supporting a model in which visual speech enhances the efficiency of auditory speech processing in pSTG. The ability to separate neural processes may make iEEG deconvolution useful for studying a variety of complex cognitive and perceptual tasks.SIGNIFICANCE STATEMENT Understanding speech is one of the most important human abilities. Speech perception uses information from both the auditory and visual modalities. It has been difficult to study neural responses to visual speech because visual-only speech is difficult or impossible to comprehend, unlike auditory-only and audiovisual speech. We used intracranial encephalography deconvolution to overcome this obstacle. We found that visual speech evokes a positive response in the human posterior superior temporal gyrus, enhancing the efficiency of auditory speech processing.
Collapse
|
23
|
Grossman S, Yeagle EM, Harel M, Espinal E, Harpaz R, Noy N, Mégevand P, Groppe DM, Mehta AD, Malach R. The Noisy Brain: Power of Resting-State Fluctuations Predicts Individual Recognition Performance. Cell Rep 2019; 29:3775-3784.e4. [DOI: 10.1016/j.celrep.2019.11.081] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2019] [Revised: 10/08/2019] [Accepted: 11/20/2019] [Indexed: 12/17/2022] Open
|
24
|
Haun A, Tononi G. Why Does Space Feel the Way it Does? Towards a Principled Account of Spatial Experience. ENTROPY 2019. [PMCID: PMC7514505 DOI: 10.3390/e21121160] [Citation(s) in RCA: 65] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
Abstract
There must be a reason why an experience feels the way it does. A good place to begin addressing this question is spatial experience, because it may be more penetrable by introspection than other qualities of consciousness such as color or pain. Moreover, much of experience is spatial, from that of our body to the visual world, which appears as if painted on an extended canvas in front of our eyes. Because it is ‘right there’, we usually take space for granted and overlook its qualitative properties. However, we should realize that a great number of phenomenal distinctions and relations are required for the canvas of space to feel ‘extended’. Here we argue that, to be experienced as extended, the canvas of space must be composed of countless spots, here and there, small and large, and these spots must be related to each other in a characteristic manner through connection, fusion, and inclusion. Other aspects of the structure of spatial experience follow from extendedness: every spot can be experienced as enclosing a particular region, with its particular location, size, boundary, and distance from other spots. We then propose an account of the phenomenal properties of spatial experiences based on integrated information theory (IIT). The theory provides a principled approach for characterizing both the quantity and quality of experience by unfolding the cause-effect structure of a physical substrate. Specifically, we show that a simple simulated substrate of units connected in a grid-like manner yields a cause-effect structure whose properties can account for the main properties of spatial experience. These results uphold the hypothesis that our experience of space is supported by brain areas whose units are linked by a grid-like connectivity. They also predict that changes in connectivity, even in the absence of changes in activity, should lead to a warping of experienced space. To the extent that this approach provides an initial account of phenomenal space, it may also serve as a starting point for investigating other aspects of the quality of experience and their physical correspondents.
Collapse
|
25
|
Grossman S, Gaziv G, Yeagle EM, Harel M, Mégevand P, Groppe DM, Khuvis S, Herrero JL, Irani M, Mehta AD, Malach R. Convergent evolution of face spaces across human face-selective neuronal groups and deep convolutional networks. Nat Commun 2019; 10:4934. [PMID: 31666525 PMCID: PMC6821842 DOI: 10.1038/s41467-019-12623-6] [Citation(s) in RCA: 54] [Impact Index Per Article: 10.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2018] [Accepted: 09/23/2019] [Indexed: 12/21/2022] Open
Abstract
The discovery that deep convolutional neural networks (DCNNs) achieve human performance in realistic tasks offers fresh opportunities for linking neuronal tuning properties to such tasks. Here we show that the face-space geometry, revealed through pair-wise activation similarities of face-selective neuronal groups recorded intracranially in 33 patients, significantly matches that of a DCNN having human-level face recognition capabilities. This convergent evolution of pattern similarities across biological and artificial networks highlights the significance of face-space geometry in face perception. Furthermore, the nature of the neuronal to DCNN match suggests a role of human face areas in pictorial aspects of face perception. First, the match was confined to intermediate DCNN layers. Second, presenting identity-preserving image manipulations to the DCNN abolished its correlation to neuronal responses. Finally, DCNN units matching human neuronal group tuning displayed view-point selective receptive fields. Our results demonstrate the importance of face-space geometry in the pictorial aspects of human face perception. Deep convolutional neural networks (DCNNs) are able to identify faces on par with humans. Here, the authors record neuronal activity from higher visual areas in humans and show that face-selective responses in the brain show similarity to those in the intermediate layers of the DCNN.
Collapse
Affiliation(s)
- Shany Grossman
- Department of Neurobiology, Weizmann Institute of Science, 76100, Rehovot, Israel
| | - Guy Gaziv
- Department of Computer Science and Applied Mathematics, Weizmann Institute of Science, 76100, Rehovot, Israel
| | - Erin M Yeagle
- Department of Neurosurgery, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell and Feinstein Institute for Medical Research, Manhasset, NY, 11030, USA
| | - Michal Harel
- Department of Neurobiology, Weizmann Institute of Science, 76100, Rehovot, Israel
| | - Pierre Mégevand
- Department of Neurosurgery, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell and Feinstein Institute for Medical Research, Manhasset, NY, 11030, USA.,Neurology Division, Clinical Neuroscience Department, Geneva University Hospital and Faculty of Medicine, Geneva, 1205, Switzerland
| | - David M Groppe
- Department of Neurosurgery, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell and Feinstein Institute for Medical Research, Manhasset, NY, 11030, USA.,The Krembil Neuroscience Centre, Toronto, ON, M5T 2S8, Canada
| | - Simon Khuvis
- Department of Neurosurgery, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell and Feinstein Institute for Medical Research, Manhasset, NY, 11030, USA
| | - Jose L Herrero
- Department of Neurosurgery, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell and Feinstein Institute for Medical Research, Manhasset, NY, 11030, USA
| | - Michal Irani
- Department of Computer Science and Applied Mathematics, Weizmann Institute of Science, 76100, Rehovot, Israel
| | - Ashesh D Mehta
- Department of Neurosurgery, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell and Feinstein Institute for Medical Research, Manhasset, NY, 11030, USA
| | - Rafael Malach
- Department of Neurobiology, Weizmann Institute of Science, 76100, Rehovot, Israel.
| |
Collapse
|
26
|
Norman Y, Yeagle EM, Khuvis S, Harel M, Mehta AD, Malach R. Hippocampal sharp-wave ripples linked to visual episodic recollection in humans. Science 2019; 365:365/6454/eaax1030. [DOI: 10.1126/science.aax1030] [Citation(s) in RCA: 91] [Impact Index Per Article: 18.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2019] [Accepted: 07/04/2019] [Indexed: 12/12/2022]
Abstract
Hippocampal sharp-wave ripples (SWRs) constitute one of the most synchronized activation events in the brain and play a critical role in offline memory consolidation. Yet their cognitive content and function during awake, conscious behavior remains unclear. We directly examined this question using intracranial recordings in human patients engaged in episodic free recall of previously viewed photographs. Our results reveal a content-selective increase in hippocampal ripple rate emerging 1 to 2 seconds prior to recall events. During recollection, high-order visual areas showed pronounced SWR-coupled reemergence of activation patterns associated with recalled content. Finally, the SWR rate during encoding predicted subsequent free-recall performance. These results point to a role for hippocampal SWRs in triggering spontaneous recollections and orchestrating the reinstatement of cortical representations during free episodic memory retrieval.
Collapse
|
27
|
Drew PJ, Winder AT, Zhang Q. Twitches, Blinks, and Fidgets: Important Generators of Ongoing Neural Activity. Neuroscientist 2019; 25:298-313. [PMID: 30311838 PMCID: PMC6800083 DOI: 10.1177/1073858418805427] [Citation(s) in RCA: 32] [Impact Index Per Article: 6.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
Animals and humans continuously engage in small, spontaneous motor actions, such as blinking, whisking, and postural adjustments ("fidgeting"). These movements are accompanied by changes in neural activity in sensory and motor regions of the brain. The frequency of these motions varies in time, is affected by sensory stimuli, arousal levels, and pathology. These fidgeting behaviors can be entrained by sensory stimuli. Fidgeting behaviors will cause distributed, bilateral functional activation in the 0.01 to 0.1 Hz frequency range that will show up in functional magnetic resonance imaging and wide-field calcium neuroimaging studies, and will contribute to the observed functional connectivity among brain regions. However, despite the large potential of these behaviors to drive brain-wide activity, these fidget-like behaviors are rarely monitored. We argue that studies of spontaneous and evoked brain dynamics in awake animals and humans should closely monitor these fidgeting behaviors. Differences in these fidgeting behaviors due to arousal or pathology will "contaminate" ongoing neural activity, and lead to apparent differences in functional connectivity. Monitoring and accounting for the brain-wide activations by these behaviors is essential during experiments to differentiate fidget-driven activity from internally driven neural dynamics.
Collapse
Affiliation(s)
- Patrick J Drew
- Department of Engineering Science and Mechanics, Pennsylvania State University, University Park, PA, USA
- Department of Neurosurgery and Department of Biomedical Engineering, Pennsylvania State University, University Park, PA, USA
| | - Aaron T Winder
- Department of Engineering Science and Mechanics, Pennsylvania State University, University Park, PA, USA
| | - Qingguang Zhang
- Department of Engineering Science and Mechanics, Pennsylvania State University, University Park, PA, USA
| |
Collapse
|
28
|
Grossman S, Gueta C, Pesin S, Malach R, Landau AN. Where Does Time Go When You Blink? Psychol Sci 2019; 30:907-916. [DOI: 10.1177/0956797619842198] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
Abstract
Retinal input is frequently lost because of eye blinks, yet humans rarely notice these gaps in visual input. Although previous studies focused on the perceptual and neural correlates of diminished awareness to blinks, the impact of these correlates on the perceived time of concurrent events is unknown. Here, we investigated whether the subjective sense of time is altered by spontaneous blinks. We found that participants ( N = 22) significantly underestimated the duration of a visual stimulus when a spontaneous blink occurred during stimulus presentation and that this underestimation was correlated with the blink duration of individual participants. Importantly, the effect was not present when durations of an auditory stimulus were judged ( N = 23). The results point to a link between spontaneous blinks, previously demonstrated to induce activity suppression in the visual cortex, and a compression of subjective time. They suggest that ongoing encoding within modality-specific sensory cortices, independent of conscious awareness, informs the subjective sense of time.
Collapse
Affiliation(s)
- Shany Grossman
- Department of Neurobiology, Weizmann Institute of Science
| | - Chen Gueta
- Department of Psychology, The Hebrew University of Jerusalem
- Department of Cognitive Sciences, The Hebrew University of Jerusalem
| | - Slav Pesin
- Department of Psychology, The Hebrew University of Jerusalem
- Department of Cognitive Sciences, The Hebrew University of Jerusalem
| | - Rafael Malach
- Department of Neurobiology, Weizmann Institute of Science
| | - Ayelet N. Landau
- Department of Psychology, The Hebrew University of Jerusalem
- Department of Cognitive Sciences, The Hebrew University of Jerusalem
| |
Collapse
|
29
|
Neuronal baseline shifts underlying boundary setting during free recall. Nat Commun 2017; 8:1301. [PMID: 29101322 PMCID: PMC5670232 DOI: 10.1038/s41467-017-01184-1] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2017] [Accepted: 08/23/2017] [Indexed: 12/16/2022] Open
Abstract
Asked to freely recall items from a predefined set (e.g., animals), we rarely recall a wrong exemplar (e.g., a vegetable). This capability is so powerful and effortless that it is essentially taken for granted, yet, surprisingly, the underlying neuronal mechanisms are unknown. Here we investigate this boundary setting mechanism using intracranial recordings (ECoG), in 12 patients undergoing epilepsy monitoring engaged in episodic free recall. After viewing vivid photographs from two categories (famous faces and places), patients were asked to freely recall these items, targeting each category in separate blocks. Our results reveal a rapid and sustained rise in neuronal activity (“baseline shift”) in high-order visual areas that persists throughout the free recall period and reflects the targeted category. We further show a more transient reactivation linked to individual recall events. The results point to baseline shift as a flexible top−down mechanism that biases spontaneous recall to remain within the required categorical boundaries. Though people are easily able to recall items in a category without mentioning a wrong exemplar, the mechanism underlying this ability is unknown. Here, authors use intracranial recordings to show that this ability is likely due to a selective increase in baseline neuronal activity in category-specific regions.
Collapse
|
30
|
Golan T, Davidesco I, Meshulam M, Groppe DM, Mégevand P, Yeagle EM, Goldfinger MS, Harel M, Melloni L, Schroeder CE, Deouell LY, Mehta AD, Malach R. Increasing suppression of saccade-related transients along the human visual hierarchy. eLife 2017; 6. [PMID: 28850030 PMCID: PMC5576487 DOI: 10.7554/elife.27819] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2017] [Accepted: 08/08/2017] [Indexed: 11/13/2022] Open
Abstract
A key hallmark of visual perceptual awareness is robustness to instabilities arising from unnoticeable eye and eyelid movements. In previous human intracranial (iEEG) work (Golan et al., 2016) we found that excitatory broadband high-frequency activity transients, driven by eye blinks, are suppressed in higher-level but not early visual cortex. Here, we utilized the broad anatomical coverage of iEEG recordings in 12 eye-tracked neurosurgical patients to test whether a similar stabilizing mechanism operates following small saccades. We compared saccades (1.3°−3.7°) initiated during inspection of large individual visual objects with similarly-sized external stimulus displacements. Early visual cortex sites responded with positive transients to both conditions. In contrast, in both dorsal and ventral higher-level sites the response to saccades (but not to external displacements) was suppressed. These findings indicate that early visual cortex is highly unstable compared to higher-level visual regions which apparently constitute the main target of stabilizing extra-retinal oculomotor influences.
Collapse
Affiliation(s)
- Tal Golan
- Edmond and Lily Safra Center for Brain Sciences, The Hebrew University of Jerusalem, Jerusalem, Israel
| | - Ido Davidesco
- Department of Psychology, New York University, New York, United States
| | - Meir Meshulam
- Department of Neurobiology, Weizmann Institute of Science, Rehovot, Israel
| | - David M Groppe
- Department of Neurosurgery, Hofstra Northwell School of Medicine, Manhasset, United States.,The Feinstein Institute for Medical Research, Manhasset, United States.,The Krembil Neuroscience Centre, Toronto, Canada
| | - Pierre Mégevand
- Department of Neurosurgery, Hofstra Northwell School of Medicine, Manhasset, United States.,The Feinstein Institute for Medical Research, Manhasset, United States
| | - Erin M Yeagle
- Department of Neurosurgery, Hofstra Northwell School of Medicine, Manhasset, United States.,The Feinstein Institute for Medical Research, Manhasset, United States
| | - Matthew S Goldfinger
- Department of Neurosurgery, Hofstra Northwell School of Medicine, Manhasset, United States.,The Feinstein Institute for Medical Research, Manhasset, United States
| | - Michal Harel
- Department of Neurobiology, Weizmann Institute of Science, Rehovot, Israel
| | - Lucia Melloni
- Department of Neurophysiology, Max Planck Institute for Brain Research, Frankfurt am Main, Germany.,NYU Comprehensive Epilepsy Center, Department of Neurology, School of Medicine, New York University, New York, United States
| | - Charles E Schroeder
- Department of Neurosurgery, Columbia University College of Physicians and Surgeons, New York, United States.,Cognitive Neuroscience and Schizophrenia Program, Nathan Kline Institute, Orangeburg, United States
| | - Leon Y Deouell
- Edmond and Lily Safra Center for Brain Sciences, The Hebrew University of Jerusalem, Jerusalem, Israel.,Department of Psychology, The Hebrew University of Jerusalem, Jerusalem, Israel
| | - Ashesh D Mehta
- Department of Neurosurgery, Hofstra Northwell School of Medicine, Manhasset, United States.,The Feinstein Institute for Medical Research, Manhasset, United States
| | - Rafael Malach
- Department of Neurobiology, Weizmann Institute of Science, Rehovot, Israel
| |
Collapse
|