1
|
Bekhelifi O, Berrached NE, Bendahmane A. Effects of the presentation order of stimulations in sequential ERP/SSVEP Hybrid Brain-Computer Interface. Biomed Phys Eng Express 2024; 10:035009. [PMID: 38430561 DOI: 10.1088/2057-1976/ad2f58] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2023] [Accepted: 03/01/2024] [Indexed: 03/04/2024]
Abstract
Hybrid Brain-Computer Interface (hBCI) combines multiple neurophysiology modalities or paradigms to speed up the output of a single command or produce multiple ones simultaneously. Concurrent hBCIs that employ endogenous and exogenous paradigms are limited by the reduced set of possible commands. Conversely, the fusion of different exogenous visual evoked potentials demonstrated impressive performances; however, they suffer from limited portability. Yet, sequential hBCIs did not receive much attention mainly due to slower transfer rate and user fatigue during prolonged BCI use (Lorenz et al 2014 J. Neural Eng. 11 035007). Moreover, the crucial factors for optimizing the hybridization remain under-explored. In this paper, we test the feasibility of sequential Event Related-Potentials (ERP) and Steady-State Visual Evoked Potentials (SSVEP) hBCI and study the effect of stimulus order presentation between ERP-SSVEP and SSVEP-ERP for the control of directions and speed of powered wheelchairs or mobile robots with 15 commands. Exploiting the fast single trial face stimulus ERP, SSVEP and modern efficient convolutional neural networks, the configuration with SSVEP presented at first achieved significantly (p < 0.05) higher average accuracy rate with 76.39% ( ± 7.30 standard deviation) hybrid command accuracy and an average Information Transfer Rate (ITR) of 25.05 ( ± 5.32 standard deviation) bits per minute (bpm). The results of the study demonstrate the suitability of a sequential SSVEP-ERP hBCI with challenging dry electroencephalography (EEG) electrodes and low-compute capacity. Although it presents lower ITR than concurrent hBCIs, our system presents an alternative in small screen settings when the conditions for concurrent hBCIs are difficult to satisfy.
Collapse
Affiliation(s)
- Okba Bekhelifi
- Intelligent Systems Research Laboratory (LARESI), Electronics Department, University of Sciences and Technology of Oran-Mohamed Boudiaf (USTO-MB), El Mnaouar, BP 1505, Bir El Djir 31000, Oran, Algeria
| | - Nasr-Eddine Berrached
- Intelligent Systems Research Laboratory (LARESI), Electronics Department, University of Sciences and Technology of Oran-Mohamed Boudiaf (USTO-MB), El Mnaouar, BP 1505, Bir El Djir 31000, Oran, Algeria
| | - Amine Bendahmane
- Signal-Image-Parole (SIMPA) Laboratory, Computer Science Department, University of Sciences and Technology of Oran-Mohamed Boudiaf (USTO-MB), El Mnaouar, BP 1505, Bir El Djir 31000, Oran, Algeria
| |
Collapse
|
2
|
Mares I, Ewing L, Papasavva M, Ducrocq E, Smith FW, Smith ML. Face recognition ability is manifest in early dynamic decoding of face-orientation selectivity-Evidence from multi-variate pattern analysis of the neural response. Cortex 2023; 159:299-312. [PMID: 36669447 DOI: 10.1016/j.cortex.2022.11.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2022] [Revised: 09/20/2022] [Accepted: 11/07/2022] [Indexed: 12/23/2022]
Abstract
Although humans are considered to be face experts, there is a well-established reliable variation in the degree to which neurotypical individuals are able to learn and recognise faces. While many behavioural studies have characterised these differences, studies that seek to relate the neuronal response to standardised behavioural measures of ability remain relatively scarce, particularly so for the time-resolved approaches and the early response to face stimuli. In the present study we make use of a relatively recent methodological advance, multi-variate pattern analysis (MVPA), to decode the time course of the neural response to faces compared to other object categories (inverted faces, objects). Importantly, for the first time, we directly relate metrics of this decoding assessed at the individual level to gold-standard measures of behavioural face processing ability assessed in an independent task. Thirty-nine participants completed the behavioural Cambridge Face Memory Test (CFMT), then viewed images of faces and houses (presented upright and inverted) while their neural activity was measured via electroencephalography. Significant decoding of both face orientation and face category were observed in all individual participants. Decoding of face orientation, a marker of more advanced face processing, was earlier and stronger in participants with higher levels of face expertise, while decoding of face category information was earlier but not stronger for individuals with greater face expertise. Taken together these results provide a marker of significant differences in the early neuronal response to faces from around 100 ms post stimulus as a function of behavioural expertise with faces.
Collapse
Affiliation(s)
- Inês Mares
- School of Psychological Science, Birkbeck College, University of London, UK; William James Center for Research, Ispa - Instituto Universitário, Portugal.
| | - Louise Ewing
- School of Psychology, University of East Anglia, Norwich, UK
| | - Michael Papasavva
- School of Psychological Science, Birkbeck College, University of London, UK
| | - Emmanuel Ducrocq
- School of Psychological Science, Birkbeck College, University of London, UK
| | - Fraser W Smith
- School of Psychology, University of East Anglia, Norwich, UK
| | - Marie L Smith
- School of Psychological Science, Birkbeck College, University of London, UK; Centre for Brain and Cognitive Development, Birkbeck College, University of London, UK
| |
Collapse
|
3
|
Shishikura M, Tamura H, Sakai K. Correlation between neural responses and human perception in figure-ground segregation. Front Syst Neurosci 2023; 16:999575. [PMID: 36713684 PMCID: PMC9877615 DOI: 10.3389/fnsys.2022.999575] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2022] [Accepted: 12/28/2022] [Indexed: 01/13/2023] Open
Abstract
Segmentation of a natural scene into objects (figures) and background (ground) is one of crucial functions for object recognition and scene understanding. Recent studies have investigated neural mechanisms underlying figure-ground (FG) segregation and reported neural modulation to FG in the intermediate-level visual area, V4, of macaque monkeys (FG neurons). However, whether FG neurons contribute to the perception of FG segregation has not been clarified. To examine the contribution of FG neurons, we examined the correlations between perceptual consistency (PC), which quantified perceptual ambiguity in FG determination, and the reliability of neural signals in response to FG. First, we evaluated PCs for the images that were used in the previous neural recording in V4; specifically, we measured how consistently FG can be determined across trials and participants for each stimulus. The PCs were widely distributed, so that we identified the ambiguity in FG segregation for each stimulus. Next, we analyzed the correlation between the PCs and the reliability of neural modulation to FG. We found that the stimuli with higher PCs evoked more consistent and greater modulation in the responses of single neurons than those with lower PCs. Since perception is expected to show a greater correlation with responses of neural population compared to those of single neurons, we examined the correlation between the PCs and the consistency of the population responses in FG determination. Stimuli with higher PCs evoked higher population consistency than those with lower PCs. Finally, we analyzed the correlation between the PCs and neural latencies in FG modulation. We found that the stimuli with higher PCs showed shorter reaction times in FG perception and evoked shorter modulation latencies in FG neurons. These results indicate that the responses of FG neurons recorded from macaque monkeys show significant correlations with human FG perception, suggesting that V4 neurons with FG-dependent responses contribute to the perception of FG segregation.
Collapse
Affiliation(s)
| | - Hiroshi Tamura
- Graduate School of Frontier Biosciences, Osaka University, Suita, Japan,Center for Information and Neural Networks, Suita, Osaka, Japan
| | - Ko Sakai
- Department of Computer Science, University of Tsukuba, Tsukuba, Japan,*Correspondence: Ko Sakai,
| |
Collapse
|
4
|
Anzures G, Mildort M. Do perceptual expertise and implicit racial bias predict early face-sensitive ERP responses? Brain Cogn 2020; 147:105671. [PMID: 33360041 DOI: 10.1016/j.bandc.2020.105671] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2020] [Revised: 11/30/2020] [Accepted: 12/04/2020] [Indexed: 11/16/2022]
Abstract
Studies examining the visual perception of face race have revealed mixed findings regarding the presence or direction of effects on early face-sensitive event-related potential (ERP) components. Few studies have examined how early ERP components are influenced by individual differences in bottom-up and top-down processes involved in face perception, and how such factors might interact to influence early face-sensitive ERP components has yet to be investigated. Thus, the current study examined whether P100, N170, and P200 responses can be predicted by individual differences in own- and other-race face recognition, implicit racial bias, and their interaction. Race effects were observed in the P100, N170, and P200 responses. Other-race face recognition, implicit racial biases, and their interaction explained a significant amount of unique variability in N170 responses when viewing other-race faces. Responses to own-race faces were minimally influenced with only implicit racial bias predicting a significant amount of unique variability in N170 latency when viewing own-race faces. Face recognition, implicit racial bias, or their interaction did not predict P100 responses. The current findings suggest that face recognition abilities and its interaction with implicit racial bias modulate the early stages of other-race face processing.
Collapse
Affiliation(s)
- Gizelle Anzures
- Department of Psychology, Florida Atlantic University, Boca Raton, FL 33431, USA; FAU Brain Institute, Florida Atlantic University, Boca Raton, FL 33431, USA; Center for Complex Systems and Brain Sciences, Florida Atlantic University, Boca Raton, FL 33431, USA.
| | - Melissa Mildort
- Department of Psychology, Florida Atlantic University, Boca Raton, FL 33431, USA
| |
Collapse
|
5
|
Nemrodov D, Behrmann M, Niemeier M, Drobotenko N, Nestor A. Multimodal evidence on shape and surface information in individual face processing. Neuroimage 2019; 184:813-825. [DOI: 10.1016/j.neuroimage.2018.09.083] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2018] [Revised: 09/22/2018] [Accepted: 09/30/2018] [Indexed: 11/27/2022] Open
|
6
|
Enhanced Fine-Form Perception Does Not Contribute to Gestalt Face Perception in Autism Spectrum Disorder. PLoS One 2017; 12:e0170239. [PMID: 28146575 PMCID: PMC5287487 DOI: 10.1371/journal.pone.0170239] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2016] [Accepted: 01/01/2017] [Indexed: 02/06/2023] Open
Abstract
Individuals with autism spectrum disorder (ASD) show superior performance in processing fine detail, but often exhibit impaired gestalt face perception. The ventral visual stream from the primary visual cortex (V1) to the fusiform gyrus (V4) plays an important role in form (including faces) and color perception. The aim of this study was to investigate how the ventral stream is functionally altered in ASD. Visual evoked potentials were recorded in high-functioning ASD adults (n = 14) and typically developing (TD) adults (n = 14). We used three types of visual stimuli as follows: isoluminant chromatic (red/green, RG) gratings, high-contrast achromatic (black/white, BW) gratings with high spatial frequency (HSF, 5.3 cycles/degree), and face (neutral, happy, and angry faces) stimuli. Compared with TD controls, ASD adults exhibited longer N1 latency for RG, shorter N1 latency for BW, and shorter P1 latency, but prolonged N170 latency, for face stimuli. Moreover, a greater difference in latency between P1 and N170, or between N1 for BW and N170 (i.e., the prolongation of cortico-cortical conduction time between V1 and V4) was observed in ASD adults. These findings indicate that ASD adults have enhanced fine-form (local HSF) processing, but impaired color processing at V1. In addition, they exhibit impaired gestalt face processing due to deficits in integration of multiple local HSF facial information at V4. Thus, altered ventral stream function may contribute to abnormal social processing in ASD.
Collapse
|
7
|
Kret ME, Tomonaga M. Getting to the Bottom of Face Processing. Species-Specific Inversion Effects for Faces and Behinds in Humans and Chimpanzees (Pan Troglodytes). PLoS One 2016; 11:e0165357. [PMID: 27902685 PMCID: PMC5130172 DOI: 10.1371/journal.pone.0165357] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2016] [Accepted: 10/11/2016] [Indexed: 11/19/2022] Open
Abstract
For social species such as primates, the recognition of conspecifics is crucial for their survival. As demonstrated by the 'face inversion effect', humans are experts in recognizing faces and unlike objects, recognize their identity by processing it configurally. The human face, with its distinct features such as eye-whites, eyebrows, red lips and cheeks signals emotions, intentions, health and sexual attraction and, as we will show here, shares important features with the primate behind. Chimpanzee females show a swelling and reddening of the anogenital region around the time of ovulation. This provides an important socio-sexual signal for group members, who can identify individuals by their behinds. We hypothesized that chimpanzees process behinds configurally in a way humans process faces. In four different delayed matching-to-sample tasks with upright and inverted body parts, we show that humans demonstrate a face, but not a behind inversion effect and that chimpanzees show a behind, but no clear face inversion effect. The findings suggest an evolutionary shift in socio-sexual signalling function from behinds to faces, two hairless, symmetrical and attractive body parts, which might have attuned the human brain to process faces, and the human face to become more behind-like.
Collapse
Affiliation(s)
- Mariska E. Kret
- Leiden University, Institute of Psychology, the Cognitive Psychology Unit, Leiden, the Netherlands
- Leiden Institute for Brain and Cognition (LIBC), Leiden, the Netherlands
| | - Masaki Tomonaga
- Primate Research Institute, Kyoto University, Inuyama, Aichi, Japan
| |
Collapse
|
8
|
Colombatto C, McCarthy G. The Effects of Face Inversion and Face Race on the P100 ERP. J Cogn Neurosci 2016; 29:664-676. [PMID: 27897676 DOI: 10.1162/jocn_a_01079] [Citation(s) in RCA: 24] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Research about the neural basis of face recognition has investigated the timing and anatomical substrates of different stages of face processing. Scalp-recorded ERP studies of face processing have focused on the N170, an ERP with a peak latency of ∼170 msec that has long been associated with the initial structural encoding of faces. However, several studies have reported earlier ERP differences related to faces, suggesting that face-specific processes might occur before N170. Here, we examined the influence of face inversion and face race on the timing of face-sensitive scalp-recorded ERPs by examining neural responses to upright and inverted line-drawn and luminance-matched white and black faces in a sample of white participants. We found that the P100 ERP evoked by inverted faces was significantly larger than that evoked by upright faces. Although this inversion effect was statistically significant at 100 msec, the inverted-upright ERP difference peaked at 138 msec, suggesting that it might represent an activity in neural sources that overlap with P100. Inverse modeling of the inversion effect difference waveform suggested possible neural sources in pericalcarine extrastriate visual cortex and lateral occipito-temporal cortex. We also found that the inversion effect difference wave was larger for white faces. These results are consistent with behavioral evidence that individuals process the faces of their own races more configurally than faces of other races. Taken together, the inversion and race effects observed in the current study suggest that configuration influences face processing by at least 100 msec.
Collapse
|