1
|
Pitt KM, McKelvey M, Weissling K, Thiessen A. Brain-computer interface for augmentative and alternative communication access: The initial training needs and learning preferences of speech-language pathologists. INTERNATIONAL JOURNAL OF SPEECH-LANGUAGE PATHOLOGY 2024:1-9. [PMID: 39028220 DOI: 10.1080/17549507.2024.2363939] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/20/2024]
Abstract
PURPOSE To enable the codesign of a training framework for brain-computer interfaces for augmentative and alternative communications access (BCI-AAC), the aim of this study is to evaluate the initial BCI-AAC training needs and preferred learning strategies of speech-language pathologists (SLPs) with AAC experience. METHOD Eleven SLPs employed across a broad range of settings completed a semi-structured interview. A grounded theory approach alongside peer debriefing and review, member checking, and triangulation procedures were utilised for thematic analysis to help ensure data reliability and credibility. RESULT Regarding critical training needs, SLPs identified the subthemes of (a) personalisation of intervention, (b) technical aspects, (c) BCI-AAC system types and access, and (d) how to support stakeholders in BCI-AAC implementation. Regarding learning strategy preferences, participants discussed (a) expert guidance and demonstrations, (b) hands-on experience, alongside (c) media and presentations. CONCLUSION Findings present a continuum of critical training needs ranging from more foundational information to more personalised assessment and intervention consideration. These thematic results present a first step in developing a basic framework for SLP training in BCI-AAC to utilise and build from as technology development continues, and provides an important initial starting point for the codesign of clinically focused BCI-AAC trainings.
Collapse
Affiliation(s)
- Kevin M Pitt
- Department of Special Education and Communication Disorders, University of Nebraska-Lincoln, Lincoln, NE, USA
| | - Miechelle McKelvey
- Department of Communication Disorders, University of Nebraska Kearney, Kearney, NE, USA
| | - Kristy Weissling
- Department of Special Education and Communication Disorders, University of Nebraska-Lincoln, Lincoln, NE, USA
| | - Amber Thiessen
- Department of Communication Sciences and Disorders, University of Houston, Houston, TX, USA
| |
Collapse
|
2
|
Pitt KM, Spoor A, Zosky J. Considering preferences, speed and the animation of multiple symbols in developing P300 brain-computer interface for children. Disabil Rehabil Assist Technol 2024:1-13. [PMID: 38808372 DOI: 10.1080/17483107.2024.2359479] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2024] [Accepted: 05/20/2024] [Indexed: 05/30/2024]
Abstract
PURPOSE Prior research has begun establishing the efficacy of animation in brain-computer interfaces access to augmentative and alternative communication (BCI-AAC). However, the use of animation in P300-BCI-AAC for children is in the early stages and largely limited to single item highlighting of extended durations. In pursuit of practical application, the present study aims to evaluate children's event-related potential (ERP) characteristics and user experience during a task involving an animated P300-BCI-AAC system. MATERIALS AND METHODS The system utilizes multi-item zoom animations to access a 28-pictorial symbols. Participants completed a fast (100 ms) and slow (200 ms) zoom animation highlighting conditions wherein four pictorial symbols were highlighted concurrently. RESULTS The proposed display appears feasible, eliciting all targeted ERPs. However, ERP amplitudes may be reduced in comparison to single-item animation highlighting, possibly due to distraction. Ratings of mental effort were significantly higher for the 100 ms condition, though differences in the frontal P200/P300 ERP did not achieve significance. Most participants identified a preference for the 100 ms condition, though age may impact preference. CONCLUSIONS Overall, findings support the preliminary feasibility of a proposed 28-item interface that utilises group zoom animation highlighting of pictorial symbols. Further research is needed evaluating ERP characteristics and outcomes from online (real-time) use of animation-based P300-BCI-AAC for children with severe speech and physical impairments across multiple training sessions.
Collapse
Affiliation(s)
- Kevin M Pitt
- Department of Special Education and Communication Disorders, University of NE-Lincoln, Lincoln, NE, USA
| | - Austin Spoor
- Department of Special Education and Communication Disorders, University of NE-Lincoln, Lincoln, NE, USA
| | - Joshua Zosky
- Department of Psychology, University of Nebraska-Lincoln, Lincoln, NE, USA
| |
Collapse
|
3
|
Pitt KM, McCarthy JW. Strategies for highlighting items within visual scene displays to support augmentative and alternative communication access for those with physical impairments. Disabil Rehabil Assist Technol 2023; 18:1319-1329. [PMID: 34788177 DOI: 10.1080/17483107.2021.2003455] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2021] [Accepted: 11/02/2021] [Indexed: 10/19/2022]
Abstract
PURPOSE In contrast to the traditional grid-based display, visual scene displays (VSDs) offer a new paradigm for aided communication. For individuals who cannot select items from an AAC display by direct selection due to physical impairments, AAC access can be supported via methods such as item scanning. Item scanning sequentially highlights items on a display until the individual signals for selection. How items are highlighted or scanned for AAC access can impact performance outcomes. Further, the effectiveness of a VSD interface may be enhanced through consultation with experts in visual communication. Therefore, to support AAC access for those with physical impairments, the aim of this study was to evaluate the perspectives of experts in visual communication regarding effective methods for highlighting VSD elements. METHODS Thirteen participants with expertise related to visual communication (e.g., photographers, artists) completed semi-structured interviews regarding techniques for item highlighting. RESULTS Study findings identified four main themes to inform how AAC items may be highlighted or scanned, including (1) use of contrast related to light and dark, (2) use of contrast as it relates to colour, (3) outline highlighting, and (4) use of scale and motion. CONCLUSION By identifying how compositional techniques can be utilized to highlight VSD elements, study findings may inform current practice for scanning-based AAC access, along with other selection techniques where feedback or highlighting is used (e.g., eye-gaze, brain-computer interface). Further, avenues for just-in-time programming are discussed to support effective implementation for those with physical impairments.IMPLICATIONS FOR REHABILITATIONFindings identify multiple potential techniques to improve scanning through items in a photograph for individuals with severe motor impairments using alternative access strategies.Study findings inform current practice for scanning-based AAC access, along with other selection techniques where feedback or highlighting is used (e.g., eye-gaze, brain-computer interface).Avenues for just in time programming of AAC displays are discussed to decrease programming demands and support effective implementation of study findings.
Collapse
Affiliation(s)
- Kevin M Pitt
- Department of Special Education and Communication Disorders, University of Nebraska-Lincoln, Lincoln, NE, USA
| | - John W McCarthy
- Division of Communication Sciences and Disorders, Ohio University, Athens, OH, USA
| |
Collapse
|
4
|
Fernández-Rodríguez Á, Ron-Angevin R, Velasco-Álvarez F, Diaz-Pineda J, Letouzé T, André JM. Evaluation of Single-Trial Classification to Control a Visual ERP-BCI under a Situation Awareness Scenario. Brain Sci 2023; 13:886. [PMID: 37371365 DOI: 10.3390/brainsci13060886] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2023] [Revised: 05/15/2023] [Accepted: 05/29/2023] [Indexed: 06/29/2023] Open
Abstract
An event-related potential (ERP)-based brain-computer interface (BCI) can be used to monitor a user's cognitive state during a surveillance task in a situational awareness context. The present study explores the use of an ERP-BCI for detecting new planes in an air traffic controller (ATC). Two experiments were conducted to evaluate the impact of different visual factors on target detection. Experiment 1 validated the type of stimulus used and the effect of not knowing its appearance location in an ERP-BCI scenario. Experiment 2 evaluated the effect of the size of the target stimulus appearance area and the stimulus salience in an ATC scenario. The main results demonstrate that the size of the plane appearance area had a negative impact on the detection performance and on the amplitude of the P300 component. Future studies should address this issue to improve the performance of an ATC in stimulus detection using an ERP-BCI.
Collapse
Affiliation(s)
- Álvaro Fernández-Rodríguez
- Departamento de Tecnología Electrónica, Instituto Universitario de Investigación en Telecomunicación de la Universidad de Málaga (TELMA), Universidad de Málaga, 29071 Malaga, Spain
| | - Ricardo Ron-Angevin
- Departamento de Tecnología Electrónica, Instituto Universitario de Investigación en Telecomunicación de la Universidad de Málaga (TELMA), Universidad de Málaga, 29071 Malaga, Spain
| | - Francisco Velasco-Álvarez
- Departamento de Tecnología Electrónica, Instituto Universitario de Investigación en Telecomunicación de la Universidad de Málaga (TELMA), Universidad de Málaga, 29071 Malaga, Spain
| | | | - Théodore Letouzé
- Laboratoire IMS, CNRS UMR 5218, Cognitive Team, Bordeaux INP-ENSC, 33400 Talence, France
| | - Jean-Marc André
- Laboratoire IMS, CNRS UMR 5218, Cognitive Team, Bordeaux INP-ENSC, 33400 Talence, France
| |
Collapse
|
5
|
Ron-Angevin R, Fernández-Rodríguez Á, Dupont C, Maigrot J, Meunier J, Tavard H, Lespinet-Najib V, André JM. Comparison of Two Paradigms Based on Stimulation with Images in a Spelling Brain-Computer Interface. SENSORS (BASEL, SWITZERLAND) 2023; 23:1304. [PMID: 36772343 PMCID: PMC9920351 DOI: 10.3390/s23031304] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/15/2022] [Revised: 01/11/2023] [Accepted: 01/20/2023] [Indexed: 06/18/2023]
Abstract
A P300-based speller can be used to control a home automation system via brain activity. Evaluation of the visual stimuli used in a P300-based speller is a common topic in the field of brain-computer interfaces (BCIs). The aim of the present work is to compare, using the usability approach, two types of stimuli that have provided high performance in previous studies. Twelve participants controlled a BCI under two conditions, which varied in terms of the type of stimulus employed: a red famous face surrounded by a white rectangle (RFW) and a range of neutral pictures (NPs). The usability approach included variables related to effectiveness (accuracy and information transfer rate), efficiency (stress and fatigue), and satisfaction (pleasantness and System Usability Scale and Affect Grid questionnaires). The results indicated that there were no significant differences in effectiveness, but the system that used NPs was reported as significantly more pleasant. Hence, since satisfaction variables should also be considered in systems that potential users are likely to employ regularly, the use of different NPs may be a more suitable option than the use of a single RFW for the development of a home automation system based on a visual P300-based speller.
Collapse
Affiliation(s)
- Ricardo Ron-Angevin
- Departamento de Tecnología Electrónica, Universidad de Málaga, 29071 Malaga, Spain
| | | | | | | | | | | | | | - Jean-Marc André
- Laboratoire IMS, CNRS UMR 5218, Cognitive Team, Bordeaux INP-ENSC, 33400 Talence, France
| |
Collapse
|
6
|
Bibliometric analysis on Brain-computer interfaces in a 30-year period. APPL INTELL 2022. [DOI: 10.1007/s10489-022-04226-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/03/2022]
|
7
|
Brain-computer interface (BCI)-generated speech to control domotic devices. Neurocomputing 2022. [DOI: 10.1016/j.neucom.2022.08.068] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
8
|
Yu X, da Silva-Sauer L, Donchin E. Habituation of P300 in the Use of P300-based Brain-Computer Interface Spellers: Individuals With Amyotrophic Lateral Sclerosis Versus Age-Matched Controls. Clin EEG Neurosci 2021; 52:221-230. [PMID: 32419492 DOI: 10.1177/1550059420918755] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
The P300-based brain-computer interface speller can provide motor independent communication to individuals with amyotrophic lateral sclerosis (ALS), a progressive neurodegenerative disorder that affects the motor system. P300 amplitude stability is critical for operation of the P300 speller. The P300 has good long-term stability, but to our knowledge, short-term habituation in the P300 speller has not been studied. In the current study, 15 participants: 8 ALS patients and 7 age-matched healthy volunteers (HVs), used 2 versions of P300 spellers, Face speller and Flash speller, each for 30 minutes. The ALS group performed as well as the HVs in both spellers and HVs did better with the Face speller than Flash speller while the ALS group performed equally well in both spellers. Neither intra-run P300 habituation nor inter-run P300 habituation was found. The P300 speller could be a reliable communication device for individuals with ALS.
Collapse
Affiliation(s)
- Xiaoqian Yu
- Department of Psychology, 7831University of South Florida, Tampa, FL, USA
| | - Leandro da Silva-Sauer
- Department of Psychology, 7831University of South Florida, Tampa, FL, USA.,123204Federal University of Paraiba, João Pessoa, Paraiba, Brazil
| | - Emanuel Donchin
- Department of Psychology, 7831University of South Florida, Tampa, FL, USA
| |
Collapse
|
9
|
Ron-Angevin R, Medina-Juliá MT, Fernández-Rodríguez Á, Velasco-Álvarez F, Andre JM, Lespinet-Najib V, Garcia L. Performance Analysis With Different Types of Visual Stimuli in a BCI-Based Speller Under an RSVP Paradigm. Front Comput Neurosci 2021; 14:587702. [PMID: 33469425 PMCID: PMC7814000 DOI: 10.3389/fncom.2020.587702] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2020] [Accepted: 11/23/2020] [Indexed: 11/28/2022] Open
Abstract
Brain-Computer Interface (BCI) systems enable an alternative communication channel for severely-motor disabled patients to interact with their environment using no muscular movements. In recent years, the importance of research into non-gaze dependent brain-computer interface paradigms has been increasing, in contrast to the most frequently studied BCI-based speller paradigm (i.e., row-column presentation, RCP). Several visual modifications that have already been validated under the RCP paradigm for communication purposes have not been validated under the most extended non-gaze dependent rapid serial visual presentation (RSVP) paradigm. Thus, in the present study, three different sets of stimuli were assessed under RSVP, with the following communication features: white letters (WL), famous faces (FF), neutral pictures (NP). Eleven healthy subjects participated in this experiment, in which the subjects had to go through a calibration phase, an online phase and, finally, a subjective questionnaire completion phase. The results showed that the FF and NP stimuli promoted better performance in the calibration and online phases, being slightly better in the FF paradigm. Regarding the subjective questionnaires, again both FF and NP were preferred by the participants in contrast to the WL stimuli, but this time the NP stimuli scored slightly higher. These findings suggest that the use of FF and NP for RSVP-based spellers could be beneficial to increase information transfer rate in comparison to the most frequently used letter-based stimuli and could represent a promising communication system for individuals with altered ocular-motor function.
Collapse
Affiliation(s)
- Ricardo Ron-Angevin
- UMA-BCI Group, Departamento de Tecnología Electrónica, Universidad de Málaga, Malaga, Spain
| | - M Teresa Medina-Juliá
- UMA-BCI Group, Departamento de Tecnología Electrónica, Universidad de Málaga, Malaga, Spain
| | | | | | - Jean-Marc Andre
- Laboratoire IMS, CNRS UMR5218, Cognitique Team, Bordeaux INP-ENSC, Talence, France
| | | | - Liliana Garcia
- Laboratoire IMS, CNRS UMR5218, Cognitique Team, Bordeaux INP-ENSC, Talence, France
| |
Collapse
|
10
|
Xu M, Han J, Wang Y, Jung TP, Ming D. Implementing Over 100 Command Codes for a High-Speed Hybrid Brain-Computer Interface Using Concurrent P300 and SSVEP Features. IEEE Trans Biomed Eng 2020; 67:3073-3082. [DOI: 10.1109/tbme.2020.2975614] [Citation(s) in RCA: 62] [Impact Index Per Article: 15.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
11
|
Philip JT, George ST. Visual P300 Mind-Speller Brain-Computer Interfaces: A Walk Through the Recent Developments With Special Focus on Classification Algorithms. Clin EEG Neurosci 2020; 51:19-33. [PMID: 30997842 DOI: 10.1177/1550059419842753] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Brain-computer interfaces are sophisticated signal processing systems, which directly operate on neuronal signals to identify specific human intents. These systems can be applied to overcome certain disabilities or to enhance the natural capabilities of human beings. The visual P300 mind-speller is a prominent one among them, which has opened up tremendous possibilities in movement and communication applications. Today, there exist many state-of-the-art visual P300 mind-speller implementations in the literature as a result of numerous researches in this domain over the past 2 decades. Each of these systems can be evaluated in terms of performance metrics like classification accuracy, information transfer rate, and processing time. Various classification techniques associated with these systems, which include but are not limited to discriminant analysis, support vector machine, neural network, distance-based and ensemble of classifiers, have major roles in determining the overall system performances. The significance of a proper review on the recent developments in visual P300 mind-spellers with proper emphasis on their classification algorithms is the key insight for this work. This article is organized with a brief introduction to P300, concepts of visual P300 mind-spellers, the survey of literature with special focus on classification algorithms, followed by the discussion of various challenges and future directions.
Collapse
Affiliation(s)
- Jobin T Philip
- Department of Electronics and Communication Engineering, Karunya Institute of Technology and Sciences, Coimbatore, Tamil Nadu, India
| | - S Thomas George
- Department of Electronics and Communication Engineering, Karunya Institute of Technology and Sciences, Coimbatore, Tamil Nadu, India
| |
Collapse
|
12
|
Evaluation of flashing stimuli shape and colour heterogeneity using a P300 brain-computer interface speller. Neurosci Lett 2019; 709:134385. [DOI: 10.1016/j.neulet.2019.134385] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2019] [Revised: 06/25/2019] [Accepted: 07/17/2019] [Indexed: 11/24/2022]
|
13
|
Zhang N, Zhou Z, Liu Y, Yin E, Jiang J, Hu D. A Novel Single-Character Visual BCI Paradigm With Multiple Active Cognitive Tasks. IEEE Trans Biomed Eng 2019; 66:3119-3128. [PMID: 30794504 DOI: 10.1109/tbme.2019.2900555] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
OBJECTIVE To introduce a novel event-related potential (ERP)-based brain-computer interface (BCI) paradigm with active mental tasks multiplying precise judgment and visual cognitive capacities and evaluate its performance. METHODS This study employed a paradigm with three types of targets (true-, pseudo-, and non-), double flash codes, colors and color terms, and four test conditions. The primary hypothesis was that active mental tasks combining multiple cognitive capacities and clear judgment for different categories of stimuli increase the BCI performance and evoke stronger or specific ERPs. Classification methods were proposed and evaluated, and two were used in online experiments. RESULTS The modes containing active mental tasks provided higher accuracy than the control mode (by up to 19.06%). The color-word matching mode had the highest judgment level and achieved the best performance. True-stimuli evoked strong P3b, while pseudotarget signals provided obvious N4, but the control mode seemed less sensitive to both of them. Different types of stimuli evoked distinctive N2 and P3a components. CONCLUSION An appropriate boost in the judgment level using multiple stimuli and cognitive approaches could be investigated to improve the BCI performance and evoke or enhance ERPs. Utilizing active mental tasks may be a promising way to promote BCIs. SIGNIFICANCE Active mental tasks combining multiple cognitive capacities and precise judgments were adopted in an ERP-based BCI. Color and color words were introduced as stimuli to construct an alternative paradigm, and the judgment levels of different conditions were calculated. High accuracies and the participants' preferences were obtained, which may promote the effective use of BCIs.
Collapse
|
14
|
Gerboni G, John SE, Rind GS, Ronayne SM, May CN, Oxley TJ, Grayden DB, Opie NL, Wong YT. Visual evoked potentials determine chronic signal quality in a stent-electrode endovascular neural interface. Biomed Phys Eng Express 2018. [DOI: 10.1088/2057-1976/aad714] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
|
15
|
Spatial and Time Domain Feature of ERP Speller System Extracted via Convolutional Neural Network. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2018; 2018:6058065. [PMID: 29861712 PMCID: PMC5976923 DOI: 10.1155/2018/6058065] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/06/2018] [Revised: 03/05/2018] [Accepted: 04/01/2018] [Indexed: 11/25/2022]
Abstract
Feature of event-related potential (ERP) has not been completely understood and illiteracy problem remains unsolved. To this end, P300 peak has been used as the feature of ERP in most brain–computer interface applications, but subjects who do not show such peak are common. Recent development of convolutional neural network provides a way to analyze spatial and temporal features of ERP. Here, we train the convolutional neural network with 2 convolutional layers whose feature maps represented spatial and temporal features of event-related potential. We have found that nonilliterate subjects' ERP show high correlation between occipital lobe and parietal lobe, whereas illiterate subjects only show correlation between neural activities from frontal lobe and central lobe. The nonilliterates showed peaks in P300, P500, and P700, whereas illiterates mostly showed peaks in around P700. P700 was strong in both subjects. We found that P700 peak may be the key feature of ERP as it appears in both illiterate and nonilliterate subjects.
Collapse
|