A Review of Emerging Access Technologies for Individuals With Severe Motor Impairments

Design of aided augmentative and alternative communication systems for children with vision impairment: psychoacoustic perspectives

Children with complex communication needs often have multiple disabilities including visual impairments that impact their ability to interact with aided augmentative and alternative communication (AAC) systems. Just as the field benefited from a consideration of visual cognitive neuroscience in construction of visual displays, an exploration of psychoacoustics can potentially assist in maximizing the possibilities within AAC systems when the visual channel is either (a) not the primary sensory mode, or (b) is one that can be augmented to ultimately benefit AAC outcomes. The purpose of this paper is to highlight background information about psychoacoustics and present possible future directions for the design of aided AAC system technologies for children with visual impairments who rely on auditory information to learn and utilize AAC.

A pediatric near-infrared spectroscopy brain-computer interface based on the detection of emotional valence

Brain-computer interfaces (BCIs) are being investigated as an access pathway to communication for individuals with physical disabilities, as the technology obviates the need for voluntary motor control. However, to date, minimal research has investigated the use of BCIs for children. Traditional BCI communication paradigms may be suboptimal given that children with physical disabilities may face delays in cognitive development and acquisition of literacy skills. Instead, in this study we explored emotional state as an alternative access pathway to communication. We developed a pediatric BCI to identify positive and negative emotional states from changes in hemodynamic activity of the prefrontal cortex (PFC). To train and test the BCI, 10 neurotypical children aged 8–14 underwent a series of emotion-induction trials over four experimental sessions (one offline, three online) while their brain activity was measured with functional near-infrared spectroscopy (fNIRS). Visual neurofeedback was used to assist participants in regulating their emotional states and modulating their hemodynamic activity in response to the affective stimuli. Child-specific linear discriminant classifiers were trained on cumulatively available data from previous sessions and adaptively updated throughout each session. Average online valence classification exceeded chance across participants by the last two online sessions (with 7 and 8 of the 10 participants performing better than chance, respectively, in Sessions 3 and 4). There was a small significant positive correlation with online BCI performance and age, suggesting older participants were more successful at regulating their emotional state and/or brain activity. Variability was seen across participants in regards to BCI performance, hemodynamic response, and discriminatory features and channels. Retrospective offline analyses yielded accuracies comparable to those reported in adult affective BCI studies using fNIRS. Affective fNIRS-BCIs appear to be feasible for school-aged children, but to further gauge the practical potential of this type of BCI, replication with more training sessions, larger sample sizes, and end-users with disabilities is necessary.

Tracking and Classification of Head Movement for Augmentative and Alternative Communication Systems

The use of assistive technologies can mitigate or reduce the challenges faced by individuals with motor disabilities to use computer systems. However, those who feature severe involuntary movements often have fewer options at hand. This work describes an application that can recognize the user’s head using a conventional webcam, track its motion, model the desired functional movement, and recognize it to enable the use of a virtual keyboard. The proposed classifier features a flexible structure and may be personalized for different user need. Experimental results obtained with participants with no neurological disorders have shown that classifiers based on Hidden Markov Models provided similar or better performance than a classifier based on position threshold. However, motion segmentation and interpretation modules were sensitive to involuntary movements featured by participants with cerebral palsy that took part in the study.

Brain-Computer interfaces for communication: preferences of individuals with locked-in syndrome, caregivers and researchers

OBJECTIVES

The development of Brain-Computer Interfaces to restore communication (cBCIs) in people with severe motor impairment ideally relies on a close collaboration between end-users and other stakeholders, such as caregivers and researchers. Awareness about potential differences in opinion between these groups is crucial for development of usable cBCIs and access technology (AT) in general. In this study, we compared the opinions of prospective cBCI users, their caregivers and cBCI researchers regarding: (1) what applications would users like to control with a cBCI; (2) what mental strategies would users prefer to use for cBCI control; and (3) at what stage of their clinical trajectory would users like to be informed about AT and cBCIs.

METHODS

We collected data from 28 individuals with locked-in syndrome, 29 of their caregivers and 28 cBCI researchers. The questionnaire was supported with animation videos to explain different cBCI concepts, the utility of which was also assessed.

RESULTS

Opinions of the three groups were aligned with respect to the most desired cBCI applications, but diverged regarding mental strategies and the timing of being informed about cBCIs. Animation videos were regarded as clear and useful tools to explain cBCIs and mental strategies to end-users and other stakeholders.

CONCLUSIONS

Disagreements were clear between stakeholders regarding which mental strategies users prefer to use and when they would like to be informed about cBCIs. To move forward in the development and clinical implementation of cBCIs, it will be necessary to align the research agendas with the needs of the end-users and caregivers.IMPLICATIONS FOR REHABILITATIONBrain-Computer Interfaces may offer people with severe motor impairment a brain-based and muscle-independent approach to control communication-technology. The successful development of communication BCIs (cBCIs) relies on a close collaboration between end-users and other stakeholders, such as caregivers and researchers.Our work reveals that people with locked-in syndrome (end-users), their caregivers and researchers developing cBCIs agree that direct and private forms of communication are the most desired cBCI applications, but disagree regarding the preferred mental strategies for cBCI control and when to be informed about cBCIs.Animation videos are an effective tool for providing information to individuals, independent of their level of health literacy, regarding the concept of cBCIs and mental strategies for control.The misalignment in opinions of different groups of stakeholders about cBCIs strengthens the argument for a user-centered design approach in the development of cBCIs and access technology designed for daily life usage.

Intelligent robot chair with communication aid using TEP responses and higher order spectra band features

In recent years, electroencephalography-based navigation and communication systems for differentially enabled communities have been progressively receiving more attention. To provide a navigation system with a communication aid, a customized protocol using thought evoked potentials has been proposed in this research work to aid the differentially enabled communities. This study presents the higher order spectra based features to categorize seven basic tasks that include Forward, Left, Right, Yes, NO, Help and Relax; that can be used for navigating a robot chair and also for communications using an oddball paradigm. The proposed system records the eight-channel wireless electroencephalography signal from ten subjects while the subject was perceiving seven different tasks. The recorded brain wave signals are pre-processed to remove the interference waveforms and segmented into six frequency band signals, i. e. Delta, Theta, Alpha, Beta, Gamma 1-1 and Gamma 2. The frequency band signals are segmented into frame samples of equal length and are used to extract the features using bispectrum estimation. Further, statistical features such as the average value of bispectral magnitude and entropy using the bispectrum field are extracted and formed as a feature set. The extracted feature sets are tenfold cross validated using multilayer neural network classifier. From the results, it is observed that the entropy of bispectral magnitude feature based classifier model has the maximum classification accuracy of 84.71 % and the value of the bispectral magnitude feature based classifier model has the minimum classification accuracy of 68.52 %.

Decoding spoken English from intracortical electrode arrays in dorsal precentral gyrus

OBJECTIVE

To evaluate the potential of intracortical electrode array signals for brain-computer interfaces (BCIs) to restore lost speech, we measured the performance of decoders trained to discriminate a comprehensive basis set of 39 English phonemes and to synthesize speech sounds via a neural pattern matching method. We decoded neural correlates of spoken-out-loud words in the 'hand knob' area of precentral gyrus, a step toward the eventual goal of decoding attempted speech from ventral speech areas in patients who are unable to speak.

APPROACH

Neural and audio data were recorded while two BrainGate2 pilot clinical trial participants, each with two chronically-implanted 96-electrode arrays, spoke 420 different words that broadly sampled English phonemes. Phoneme onsets were identified from audio recordings, and their identities were then classified from neural features consisting of each electrode's binned action potential counts or high-frequency local field potential power. Speech synthesis was performed using the 'Brain-to-Speech' pattern matching method. We also examined two potential confounds specific to decoding overt speech: acoustic contamination of neural signals and systematic differences in labeling different phonemes' onset times.

MAIN RESULTS

A linear decoder achieved up to 29.3% classification accuracy (chance = 6%) across 39 phonemes, while an RNN classifier achieved 33.9% accuracy. Parameter sweeps indicated that performance did not saturate when adding more electrodes or more training data, and that accuracy improved when utilizing time-varying structure in the data. Microphonic contamination and phoneme onset differences modestly increased decoding accuracy, but could be mitigated by acoustic artifact subtraction and using a neural speech onset marker, respectively. Speech synthesis achieved r = 0.523 correlation between true and reconstructed audio.

SIGNIFICANCE

The ability to decode speech using intracortical electrode array signals from a nontraditional speech area suggests that placing electrode arrays in ventral speech areas is a promising direction for speech BCIs.

Cursor Click Modality in an Accelerometer-Based Computer Access Device

The purpose of this study is to investigate the effects of different cursor click modalities in an alternative computer access device using accelerometry from head tilt to control cursor movement. Eighteen healthy adults performed a target acquisition task using the device with five different cursor click modalities, while maintaining cursor movement control via accelerometry. Three dwell-based click modalities with dwell times of 0.5 s, 1.0 s, and 1.5 s were tested. Two surface electromyography-based click modalities - with the sensor placed next to the eye for a blink and above the eyebrow for a brow raise - were tested. Performance was evaluated using metrics of target selection accuracy, path efficiency, target selection time, and user effort. Surface electromyography-based click modalities were as fast as the shortest dwell time and as accurate as the longest dwell time, and also minimized user effort. Three of the four performance metrics were not affected by sensor location. Future studies will investigate if these results are similar in individuals with neuromuscular disorders.

Augmentative and Alternative Communication (AAC) Advances: A Review of Configurations for Individuals with a Speech Disability

High-tech augmentative and alternative communication (AAC) methods are on a constant rise; however, the interaction between the user and the assistive technology is still challenged for an optimal user experience centered around the desired activity. This review presents a range of signal sensing and acquisition methods utilized in conjunction with the existing high-tech AAC platforms for individuals with a speech disability, including imaging methods, touch-enabled systems, mechanical and electro-mechanical access, breath-activated methods, and brain–computer interfaces (BCI). The listed AAC sensing modalities are compared in terms of ease of access, affordability, complexity, portability, and typical conversational speeds. A revelation of the associated AAC signal processing, encoding, and retrieval highlights the roles of machine learning (ML) and deep learning (DL) in the development of intelligent AAC solutions. The demands and the affordability of most systems hinder the scale of usage of high-tech AAC. Further research is indeed needed for the development of intelligent AAC applications reducing the associated costs and enhancing the portability of the solutions for a real user’s environment. The consolidation of natural language processing with current solutions also needs to be further explored for the amelioration of the conversational speeds. The recommendations for prospective advances in coming high-tech AAC are addressed in terms of developments to support mobile health communicative applications.

Speaking Ability while Using an Inductive Tongue-Computer Interface for Individuals with Tetraplegia: Talking and Driving a Powered Wheelchair - a Case Study

This paper assesses the ability of speaking while using an inductive tongue-computer interface. Lately, tongue- computer interfaces have been proposed for computer/robotic interfacing for individuals with tetraplegia. To be useful in home settings these interfaces should be aesthetic and interfere as little as possible with the limited preserved functionality of individuals with tetraplegia. As tongue interfaces from an aesthetical point of view are preferred to be entirely intra-oral it is relevant to address their effect on speech. Here we show that reading more than 566 words while using an inductive tongue-computer interface results in a maximum sensor activation time of less than 0.6 s, which means that false activations can be avoided by a sensor dwell time of 0.6 s. Furthermore, we show that it is possible to speak while controlling a powered wheelchair with the inductive tongue computer interface.

An EOG-Based Human-Machine Interface to Control a Smart Home Environment for Patients With Severe Spinal Cord Injuries

OBJECTIVE

This paper presents an asyn-chronous electrooculography (EOG)-based human-machine interface (HMI) for smart home environmental control with the purpose of providing daily assistance for severe spinal cord injury (SCI) patients.

METHODS

The proposed HMI allows users to interact with a smart home environment through eye blinking. Specifically, several buttons, each corresponding to a control command, randomly flash on a graphical user interface. Each flash of the buttons functions as a visual cue for the user to blink. To issue a control command, the user can blink synchronously with the flashes of the corresponding button. Through detecting blinks based on the recorded EOG signal, the target button and its corresponding control command are determined. Seven SCI patients participated in an online experiment, during which the patients were required to control a smart home environment including household electrical appliances, an intelligent wheelchair, as well as a nursing bed via the proposed HMI.

RESULTS

The average false operation ratio in the control state was 4.1%, whereas during the idle state, no false operations occurred.

CONCLUSION

All SCI patients were able to control the smart home environment using the proposed EOG-based HMI with satisfactory performance in terms of the false operation ratio in both the control and the idle states.

SIGNIFICANCE

The proposed HMI offers a simple and effective approach for patients with severe SCIs to control a smart home environment. Therefore, it is promising to assist severe SCI patients in their daily lives.

A qualitative study adopting a user-centered approach to design and validate a brain computer interface for cognitive rehabilitation for people with brain injury

Cognitive rehabilitation is established as a core intervention within rehabilitation programs following a traumatic brain injury (TBI). Digitally enabled assistive technologies offer opportunities for clinicians to increase remote access to rehabilitation supporting transition into home. Brain Computer Interface (BCI) systems can harness the residual abilities of individuals with limited function to gain control over computers through their brain waves. This paper presents an online cognitive rehabilitation application developed with therapists, to work remotely with people who have TBI, who will use BCI at home to engage in the therapy. A qualitative research study was completed with people who are community dwellers post brain injury (end users), and a cohort of therapists involved in cognitive rehabilitation. A user-centered approach over three phases in the development, design and feasibility testing of this cognitive rehabilitation application included two tasks (Find-a-Category and a Memory Card task). The therapist could remotely prescribe activity with different levels of difficulty. The service user had a home interface which would present the therapy activities. This novel work was achieved by an international consortium of academics, business partners and service users.

Error-Free Text Typing Performance of an Inductive Intra-Oral Tongue Computer Interface for Severely Disabled Individuals

For severely paralyzed individuals, alternative computer interfaces are becoming increasingly essential for everyday life as social and vocational activities are facilitated by information technology and as the environment becomes more automatic and remotely controllable. Tongue computer interfaces have proven to be desirable by the users partly due to their high degree of aesthetic acceptability, but so far the mature systems have shown a relatively low error-free text typing efficiency. This paper evaluated the intra-oral inductive tongue computer interface (ITCI) in its intended use: Error-free text typing in a generally available text editing system, Word. Individuals with tetraplegia and able bodied individuals used the ITCI for typing using a MATLAB interface and for Word typing for 4 to 5 experimental days, and the results showed an average error-free text typing rate in Word of 11.6 correct characters/min across all participants and of 15.5 correct characters/min for participants familiar with tongue piercings. Improvements in typing rates between the sessions suggest that typing ratescan be improved further through long-term use of the ITCI.

Challenges of implementing a personalized mental task near-infrared spectroscopy brain-computer interface for a non-verbal young adult with motor impairments

PURPOSE

Near-infrared spectroscopy brain-computer interfaces (NIRS-BCIs) have been proposed as potential motor-free communication pathways. This paper documents the challenges of implementing an NIRS-BCI with a non-verbal, severely and congenitally impaired, but cognitively intact young adult.

METHODS

A 5-session personalized mental task NIRS-BCI training paradigm was invoked, whereby participant-specific mental tasks were selected either by the researcher or by the user, on the basis of prior performance or user preference.

RESULTS

Although the personalized mental task selection and training framework had been previously demonstrated with able-bodied participants, the participant was not able to exceed chance-level accuracies. Challenges to the acquisition of BCI control may have included disinclination to BCI training, structural or functional brain atypicalities, heightened emotional arousal and confounding haemodynamic patterns associated with novelty and reward processing.

CONCLUSIONS

Overall, we stress the necessity for further clinical NIRS-BCI research involving non-verbal individuals with severe motor impairments.

Development and functional demonstration of a wireless intraoral inductive tongue computer interface for severely disabled persons

PURPOSE

Individuals with tetraplegia depend on alternative interfaces in order to control computers and other electronic equipment. Current interfaces are often limited in the number of available control commands, and may compromise the social identity of an individual due to their undesirable appearance. The purpose of this study was to implement an alternative computer interface, which was fully embedded into the oral cavity and which provided multiple control commands.

METHODS

The development of a wireless, intraoral, inductive tongue computer was described. The interface encompassed a 10-key keypad area and a mouse pad area. This system was embedded wirelessly into the oral cavity of the user. The functionality of the system was demonstrated in two tetraplegic individuals and two able-bodied individuals Results: The system was invisible during use and allowed the user to type on a computer using either the keypad area or the mouse pad. The maximal typing rate was 1.8 s for repetitively typing a correct character with the keypad area and 1.4 s for repetitively typing a correct character with the mouse pad area.

CONCLUSION

The results suggest that this inductive tongue computer interface provides an esthetically acceptable and functionally efficient environmental control for a severely disabled user. Implications for Rehabilitation New Design, Implementation and detection methods for intra oral assistive devices. Demonstration of wireless, powering and encapsulation techniques suitable for intra oral embedment of assistive devices. Demonstration of the functionality of a rechargeable and fully embedded intra oral tongue controlled computer input device.

Application of an access technology delivery protocol to two children with cerebral palsy

PURPOSE

This study further delineates the merits and limitations of the Access Technology Delivery Protocol (ATDP) through its application to two children with severe disabilities.

METHOD

We conducted mixed methods case studies to demonstrate the ATDP with two children with no reliable means of access to an external device. Evaluations of response efficiency, satisfaction, goal attainment, technology use and participation were made after 8 and 16 weeks of training with custom access technologies.

RESULTS

After 16 weeks, one child's switch offered improved response efficiency, high teacher satisfaction and increased participation. The other child's switch resulted in improved satisfaction and switch effectiveness but lower overall efficiency. The latter child was no longer using his switch by the end of the study.

CONCLUSIONS

These contrasting findings indicate that changes to any contextual factors that may impact the user's switch performance should mandate a reassessment of the access pathway. Secondly, it is important to ensure that individuals who will be responsible for switch training be identified at the outset and engaged throughout the ATDP. Finally, the ATDP should continue to be tested with individuals with severe disabilities to build an evidence base for the delivery of response efficient access solutions. Implications for Rehabilitation A data-driven, comprehensive access technology delivery protocol for children with complex communication needs could help to mitigate technology abandonment. Successful adoption of an access technology requires personalized design, training of the technology user, the teaching staff, the caregivers and other communication partners, and integration with functional activities.

Single-Trial Analysis of Inter-Beat Interval Perturbations Accompanying Single-Switch Scanning: Case Series of Three Children With Severe Spastic Quadriplegic Cerebral Palsy

Single-switch access in conjunction with scanning remains a fundamental solution in restoring communication for many children with profound physical disabilities. However, untimely switch inaction and unintentional switch activations can lead to user frustration and impede functional communication. A previous preliminary study, in the context of a case series with three single-switch users, reported that correct, accidental and missed switch activations could elicit cardiac deceleration and increased phasic skin conductance on average, while deliberate switch non-use was associated with autonomic nonresponse. The present study investigated the possibility of using blood volume pulse recordings from the same three pediatric single-switch users to track the aforementioned switch events on a single-trial basis. Peaks of the line length time series derived from the empirical mode decomposition of the inter-beat interval time series matched, on average, a high percentage (above 80%) of single-switch events, while unmatched peaks coincided moderately (below 37%) with idle time during scanning. These results encourage further study of autonomic measures as complementary information channels to enhance single-switch access.

Effects of response-related music stimulation versus general music stimulation on positive participation of patients with Alzheimer's disease

OBJECTIVE

Assessing the effects of response-related music stimulation versus general (response-unrelated) music stimulation on positive participation of 11 new patients with Alzheimer's disease.

METHOD

The patients were functioning in the severe and low-moderate ranges of the disease. Positive participation included behaviors such as, singing or rhythmic movements and smiles. Both music conditions relied on the display of music/song videos on a computer screen. In the response-related (active) condition, the patients used a simple hand response and a microswitch to determine music stimulation inputs. In the general (unrelated/passive) condition, music stimulation was automatically presented throughout the sessions.

RESULTS

Data showed that six of the 11 patients had higher levels of positive participation in the response-related stimulation condition. The remaining five patients did not have differences between the two conditions.

CONCLUSION

Based on this evidence and previous findings, one might consider the use of the active condition beneficial for daily programs.

Training to use a commercial brain-computer interface as access technology: a case study

PURPOSE

This case study describes how an individual with spastic quadriplegic cerebral palsy was trained over a period of four weeks to use a commercial electroencephalography (EEG)-based brain-computer interface (BCI).

METHOD

The participant spent three sessions exploring the system, and seven sessions playing a game focused on EEG feedback training of left and right arm motor imagery and a customised, training game paradigm was employed.

RESULTS

The participant showed improvement in the production of two distinct EEG patterns. The participant's performance was influenced by motivation, fatigue and concentration. Six weeks post-training the participant could still control the BCI and used this to type a sentence using an augmentative and alternative communication application on a wirelessly linked device.

CONCLUSIONS

The results from this case study highlight the importance of creating a dynamic, relevant and engaging training environment for BCIs. Implications for Rehabilitation Customising a training paradigm to suit the users' interests can influence adherence to assistive technology training. Mood, fatigue, physical illness and motivation influence the usability of a brain-computer interface. Commercial brain-computer interfaces, which require little set up time, may be used as access technology for individuals with severe disabilities.

User modeling for people with special needs

Purpose

– The purpose of this paper is to present an approach where a novel user modeling wizard for people with motor impairments is used to gain a deeper understanding of very specific (touch-based and touchless) interaction patterns. The findings are used to set up and fill a user model which allows to automatically derive an application- and user-specific configuration for natural user interfaces.

Design/methodology/approach

– Based on expert knowledge in the domain of software/user interfaces for people with special needs, a test-case –based user modeling tool was developed. Task-based user tests were conducted with seven users for the touch-based interaction scenario and with five users for the touchless interaction scenario. The participants are all people with different motor and/or cognitive impairments.

Findings

– The paper describes the results of different test cases that were designed to model users’ touch-based and touchless interaction capabilities. To evaluate the tool’s findings, experts additionally judged the participants’ performance (their opinions were compared to the tool’s findings). The results suggest that the user modeling tool could quite well capture users’ capabilities.

Social implications

– The paper presents a tool that can be used to model users’ interaction capabilities. The approach aims at taking over some of the (very time-consuming) configuration tasks consultants have to do to configure software according to the needs of people with disabilities. This can lead to a wider accessibility of software, especially in the area of gesture-based user interaction.

Originality/value

– Part of the approach has been published in the proceedings of the Interactional Conference on Advances in Mobile Computing and Multimedia 2014. Significant additions have been made since (e.g. all of the touchless interaction part of the approach and the related user study).

Trends in communicative access solutions for children with cerebral palsy

Access solutions may facilitate communication in children with limited functional speech and motor control. This study reviews current trends in access solution development for children with cerebral palsy, with particular emphasis on the access technology that harnesses a control signal from the user (eg, movement or physiological change) and the output device (eg, augmentative and alternative communication system) whose behavior is modulated by the user's control signal. Access technologies have advanced from simple mechanical switches to machine vision (eg, eye-gaze trackers), inertial sensing, and emerging physiological interfaces that require minimal physical effort. Similarly, output devices have evolved from bulky, dedicated hardware with limited configurability, to platform-agnostic, highly personalized mobile applications. Emerging case studies encourage the consideration of access technology for all nonverbal children with cerebral palsy with at least nascent contingency awareness. However, establishing robust evidence of the effectiveness of the aforementioned advances will require more expansive studies.

Towards a physiological signal-based access solution for a non-verbal adolescent with severe and multiple disabilities

OBJECTIVE

To find physiologically arousing stimuli and labile physiological channels in a non-verbal adolescent with severe and multiple congenital disabilities, who did not have a reliable means of communication.

METHODS

The client was repeatedly presented with visual and audiovisual stimuli, representing variations of six contextual factors over three sessions in a one month period. For each stimulus, reactions were detected in the client's four peripheral autonomic nervous system signals using a rule-based classification algorithm.

RESULTS

During the presentation of audiovisual stimuli, the number of physiological reactions significantly differed from that observed in baseline (χ(2) = 3.93, p = 0.0476). Aural stimuli articulated in an unfamiliar voice, and aural stimuli containing anticipatory patterns were also physiologically arousing. Fingertip temperature was the client's most labile physiological signal.

CONCLUSIONS

The results of this case study suggest that physiological data may complement caregiver acumen in deciphering the reactions of non-verbal clients with severe and multiple disabilities.

An access technology delivery protocol for children with severe and multiple disabilities: a case demonstration

OBJECTIVE

This study applied response efficiency theory to create the Access Technology Delivery Protocol (ATDP), a child and family-centred collaborative approach to the implementation of access technologies.

METHODS

We conducted a descriptive, mixed methods case study to demonstrate the ATDP method with a 12-year-old boy with no reliable means of access to an external device. Evaluations of response efficiency, satisfaction, goal attainment, technology use and participation were made after 8 and 16 weeks of training with a custom smile-based access technology.

RESULTS

At the 16 week mark, the new access technology offered better response quality; teacher satisfaction was high; average technology usage was 3-4 times per week for up to 1 h each time; switch sensitivity and specificity reached 78% and 64%, respectively, and participation scores increased by 38%.

CONCLUSION

This case supports further development and testing of the ATDP with additional children with multiple or severe disabilities.

New camera-based microswitch technology to monitor small head and mouth responses of children with multiple disabilities

OBJECTIVE

Assessing a new camera-based microswitch technology, which did not require the use of color marks on the participants' face.

METHOD

Two children with extensive multiple disabilities participated. The responses selected for them consisted of small, lateral head movements and mouth closing or opening. The intervention was carried out according to a multiple probe design across responses. The technology involved a computer with a CPU using a 2-GHz clock, a USB video camera with a 16-mm lens, a USB cable connecting the camera and the computer, and a special software program written in ISO C++ language.

RESULTS

The new technology was satisfactorily used with both children. Large increases in their responding were observed during the intervention periods (i.e. when the responses were followed by preferred stimulation).

CONCLUSION

The new technology may be an important resource for persons with multiple disabilities and minimal motor behavior.

Online transcranial Doppler ultrasonographic control of an onscreen keyboard

Brain-computer interface (BCI) systems exploit brain activity for generating a control command and may be used by individuals with severe motor disabilities as an alternative means of communication. An emerging brain monitoring modality for BCI development is transcranial Doppler ultrasonography (TCD), which facilitates the tracking of cerebral blood flow velocities associated with mental tasks. However, TCD-BCI studies to date have exclusively been offline. The feasibility of a TCD-based BCI system hinges on its online performance. In this paper, an online TCD-BCI system was implemented, bilaterally tracking blood flow velocities in the middle cerebral arteries for system-paced control of a scanning keyboard. Target letters or words were selected by repetitively rehearsing the spelling while imagining the writing of the intended word, a left-lateralized task. Undesired letters or words were bypassed by performing visual tracking, a non-lateralized task. The keyboard scanning period was 15 s. With 10 able-bodied right-handed young adults, the two mental tasks were differentiated online using a Naïve Bayes classification algorithm and a set of time-domain, user-dependent features. The system achieved an average specificity and sensitivity of 81.44 ± 8.35 and 82.30 ± 7.39%, respectively. The level of agreement between the intended and machine-predicted selections was moderate (κ = 0.60). The average information transfer rate was 0.87 bits/min with an average throughput of 0.31 ± 0.12 character/min. These findings suggest that an online TCD-BCI can achieve reasonable accuracies with an intuitive language task, but with modest throughput. Future interface and signal classification enhancements are required to improve communication rate.

Autonomic responses to correct outcomes and interaction errors during single-switch scanning among children with severe spastic quadriplegic cerebral palsy

BACKGROUND

The combination of single-switch access technology and scanning is the most promising means of augmentative and alternative communication for many children with severe physical disabilities. However, the physical impairment of the child and the technology's limited ability to interpret the child's intentions often lead to false positives and negatives (corresponding to accidental and missed selections, respectively) occurring at rates that frustrate the user and preclude functional communication. Multiple psychophysiological studies have associated cardiac deceleration and increased phasic electrodermal activity with self-realization of errors among able-bodied individuals. Thus, physiological measurements have potential utility at enhancing single-switch access, provided that such prototypical autonomic responses exist in persons with profound disabilities.

METHODS

The present case series investigated the autonomic responses of three pediatric single-switch users with severe spastic quadriplegic cerebral palsy, in the context of a single-switch letter matching activity. Each participant exhibited distinct autonomic responses to activity engagement.

RESULTS

Our analysis confirmed the presence of the autonomic response pattern of cardiac deceleration and increased phasic electrodermal activity following true positives, false positives and false negatives errors, but not subsequent to true negative outcomes.

CONCLUSIONS

These findings suggest that there may be merit in complementing single-switch input with autonomic measurements to improve augmentative and alternative communications for pediatric access technology users.

Three non-ambulatory adults with multiple disabilities exercise foot-leg movements through microswitch-aided programs

This study assessed the use of microswitch-aided programs to help three non-ambulatory adults with multiple disabilities exercise foot-leg responses. Those responses served to activate a largely neglected part of the participants' body, with possibly positive physical implications (e.g., for blood circulation, swelling, and muscle strength). Intervention focused on the left and right foot-leg response, separately. Eventually, sessions with one response were alternated with sessions with the other response. Responses were monitored via microswitches and followed by 8s of preferred stimulation (e.g., music and vibrotactile stimulation), which was automatically delivered. The results showed that all three participants had high levels of foot-leg responses during the intervention phases and a 3-week post-intervention check. The participants also displayed expressions of positive involvement during those study periods (i.e., engaged in behaviors, such as music-related head movements, smiles, or touching the vibratory devices) that could be interpreted as forms of interest/pleasure and happiness. These results are in line with previous findings in this area and can be taken as an important confirmation of the strength and dependability of the approach in motivating non-ambulatory persons with multiple disabilities to engage in foot-leg movements. The practical implications of these findings are discussed.

Self-regulated music stimulation for persons with Alzheimer's disease: impact assessment and social validation

OBJECTIVE

To assess the impact and the social rating of an active music condition (in which 10 patients with Alzheimer's disease regulated their music input) vs. a passive music condition.

METHOD

In the active condition, the patients used a simple hand response and a microswitch to activate music stimulation periods. In the passive condition, music stimulation was prearranged and continued through the sessions. The active and passive stimulation sessions were preceded and followed by control (non-stimulation) sessions.

RESULTS

The active condition sessions showed an increase in the patients' indices of positive participation (e.g., singing or music-related movements, and smiles) similar to that observed in the passive condition sessions. Social raters (140 psychology students) favored the active condition on a six-item questionnaire dealing, among others, with conditions' suitability, respect of patients' dignity and independence, and practicality.

CONCLUSION

An active music stimulation condition can be viable, effective, and socially preferable.

Performance assessment in brain-computer interface-based augmentative and alternative communication

A large number of incommensurable metrics are currently used to report the performance of brain-computer interfaces (BCI) used for augmentative and alterative communication (AAC). The lack of standard metrics precludes the comparison of different BCI-based AAC systems, hindering rapid growth and development of this technology. This paper presents a review of the metrics that have been used to report performance of BCIs used for AAC from January 2005 to January 2012. We distinguish between Level 1 metrics used to report performance at the output of the BCI Control Module, which translates brain signals into logical control output, and Level 2 metrics at the Selection Enhancement Module, which translates logical control to semantic control. We recommend that: (1) the commensurate metrics Mutual Information or Information Transfer Rate (ITR) be used to report Level 1 BCI performance, as these metrics represent information throughput, which is of interest in BCIs for AAC; 2) the BCI-Utility metric be used to report Level 2 BCI performance, as it is capable of handling all current methods of improving BCI performance; (3) these metrics should be supplemented by information specific to each unique BCI configuration; and (4) studies involving Selection Enhancement Modules should report performance at both Level 1 and Level 2 in the BCI system. Following these recommendations will enable efficient comparison between both BCI Control and Selection Enhancement Modules, accelerating research and development of BCI-based AAC systems.

Towards a multimodal brain-computer interface: combining fNIRS and fTCD measurements to enable higher classification accuracy

Previous brain-computer interface (BCI) research has largely focused on single neuroimaging modalities such as near-infrared spectroscopy (NIRS) or transcranial Doppler ultrasonography (TCD). However, multimodal brain-computer interfaces, which combine signals from different brain modalities, have been suggested as a potential means of improving the accuracy of BCI systems. In this paper, we compare the classification accuracies attainable using NIRS signals alone, TCD signals alone, and a combination of NIRS and TCD signals. Nine able-bodied subjects (mean age=25.7) were recruited and simultaneous measurements were made with NIRS and TCD instruments while participants were prompted to perform a verbal fluency task or to remain at rest, within the context of a block-stimulus paradigm. Using Linear Discriminant Analysis, the verbal fluency task was classified at mean accuracies of 76.1±9.9%, 79.4±10.3%, and 86.5±6.0% using NIRS, TCD, and NIRS-TCD systems respectively. In five of nine participants, classification accuracies with the NIRS-TCD system were significantly higher (p<0.05) than with NIRS or TCD systems alone. Our results suggest that multimodal neuroimaging may be a promising method of improving the accuracy of future brain-computer interfaces.

Assessing the impact and social perception of self-regulated music stimulation with patients with Alzheimer's disease

We assessed the impact and social rating of an active and a passive music condition implemented with six patients with Alzheimer's disease. In the active condition, the patients used a simple hand response and a microswitch to self-regulate music stimulation inputs. In the passive condition, music stimulation was automatically presented throughout the sessions. Active and passive stimulation sessions were preceded and followed by control (non-stimulation) sessions. The active condition sessions showed an increase in the patients' indices of positive participation (e.g., singing or music-related movements, and smiles) greater than that observed in the passive condition sessions for five of the six patients. Positive intervention effects could also spread to the post-intervention sessions. Social raters (42 care and rehabilitation staff members working with persons with multiple disabilities) favored the active condition on a six-item questionnaire dealing with, among others, conditions' suitability, respect of patients' dignity and independence, and practicality. The implications of the findings as to the plausibility/desirability of an active stimulation condition were discussed.

Towards a hemodynamic BCI using transcranial Doppler without user-specific training data

Transcranial Doppler (TCD) was recently introduced as a new brain-computer interface (BCI) modality for detecting task-induced hemispheric lateralization. To date, single-trial discrimination between a lateralized mental activity and a rest state has been demonstrated with long (45 s) activation time periods. However, the possibility of detecting successive activations in a user-independent framework (i.e. without training data from the user) remains an open question.

OBJECTIVE

The objective of this research was to assess TCD-based detection of lateralized mental activity with a user-independent classifier. In so doing, we also investigated the accuracy of detecting successive lateralizations. Approach. TCD data from 18 participants were collected during verbal fluency, mental rotation tasks and baseline counting tasks. Linear discriminant analysis and a set of four time-domain features were used to classify successive left and right brain activations.

MAIN RESULTS

In a user-independent framework, accuracies up to 74.6 ± 12.6% were achieved using training data from a single participant, and lateralization task durations of 18 s.

SIGNIFICANCE

Subject-independent, algorithmic classification of TCD signals corresponding to successive brain lateralization may be a feasible paradigm for TCD-BCI design.

Challenges of developing communicative interaction in individuals with congenital profound intellectual and multiple disabilities

BACKGROUND

Physiological responses have been used in individuals with acquired disability to enable communicative interaction without motor movement. This study explored four autonomic nervous system (ANS) signals-electrodermal activity, skin temperature, cardiac patterns and respiratory patterns-to enable interaction with individuals born with profound intellectual and multiple disabilities (PIMD).

METHOD

A series of case studies were conducted to teach a 15-year-old individual with PIMD to voluntarily control his ANS signals for the purposes of communicative interaction. Training was carried out according to an alternating treatment single-subject research design.

RESULTS

Training was unsuccessful; however, ANS signal patterns revealed issues unique to people with PIMD: profound intrasubject variability, discrepancy between physiological responses and caregiver perspectives, and the participant's lack of contingency awareness.

CONCLUSIONS

These three priority areas unique to people with congenital PIMD must be addressed before ANS signals can be used to enable communicative interaction with this population.

Two persons with multiple disabilities use camera-based microswitch technology to control stimulation with small mouth and eyelid responses

BACKGROUND

A camera-based microswitch technology was recently developed to monitor small facial responses of persons with multiple disabilities and allow those responses to control environmental stimulation. This study assessed such a technology with 2 new participants using slight variations of previous responses.

METHOD

The technology involved a computer with a CPU using a 2GHz clock, a USB video camera with 16-mm lens, and special software. Small colour spots were used under the lower lip of one participant and on the eyelid of the other participant to aid the camera and computer to detect their mouth and eyelid responses. The study involved an ABAB design and included a 3-week post-intervention check.

RESULTS

The participants' mouth and eyelid responses increased during the intervention (B) phases and post-intervention check (i.e., when the technology allowed them to control stimulation).

CONCLUSIONS

Camera-based microswitch technology can help persons with multiple disabilities control stimulation with small responses.

Towards increased data transmission rate for a three-class metabolic brain-computer interface based on transcranial Doppler ultrasound

In this study, we conducted an offline analysis of transcranial Doppler (TCD) ultrasound recordings to investigate potential methods for increasing data transmission rate in a TCD-based brain-computer interface. Cerebral blood flow velocity was recorded within the left and right middle cerebral arteries while nine able-bodied participants alternated between rest and two different mental activities (word generation and mental rotation). We differentiated these three states using a three-class linear discriminant analysis classifier while the duration of each state was varied between 5 and 30s. Maximum classification accuracies exceeded 70%, and data transmission rate was maximized at 1.2 bits per minute, representing a four-fold increase in data transmission rate over previous two-class analysis of TCD recordings.

Automatic single-trial discrimination of mental arithmetic, mental singing and the no-control state from prefrontal activity: toward a three-state NIRS-BCI

Background

Near-infrared spectroscopy (NIRS) is an optical imaging technology that has recently been investigated for use in a safe, non-invasive brain-computer interface (BCI) for individuals with severe motor impairments. To date, most NIRS-BCI studies have attempted to discriminate two mental states (e.g., a mental task and rest), which could potentially lead to a two-choice BCI system. In this study, we attempted to automatically differentiate three mental states - specifically, intentional activity due to 1) a mental arithmetic (MA) task and 2) a mental singing (MS) task, and 3) an unconstrained, "no-control (NC)" state - to investigate the feasibility of a three-choice system-paced NIRS-BCI.

Results

Deploying a dual-wavelength frequency domain near-infrared spectrometer, we interrogated nine sites around the frontopolar locations while 7 able-bodied adults performed mental arithmetic and mental singing to answer multiple-choice questions within a system-paced paradigm. With a linear classifier trained on a ten-dimensional feature set, an overall classification accuracy of 56.2% was achieved for the MA vs. MS vs. NC classification problem and all individual participant accuracies significantly exceeded chance (i.e., 33%). However, as anticipated based on results of previous work, the three-class discrimination was unsuccessful for three participants due to the ineffectiveness of the mental singing task. Excluding these three participants increases the accuracy rate to 62.5%. Even without training, three of the remaining four participants achieved accuracies approaching 70%, the value often cited as being necessary for effective BCI communication.

Conclusions

These results are encouraging and demonstrate the potential of a three-state system-paced NIRS-BCI with two intentional control states corresponding to mental arithmetic and mental singing.

Classification of activity engagement in individuals with severe physical disabilities using signals of the peripheral nervous system

Communication barriers often result in exclusion of children and youth with disabilities from activities and social settings that are essential to their psychosocial development. In particular, difficulties in describing their experiences of activities and social settings hinder our understanding of the factors that promote inclusion and participation of this group of individuals. To address this specific communication challenge, we examined the feasibility of developing a language-free measure of experience in youth with severe physical disabilities. To do this, we used the activity of the peripheral nervous system to detect patterns of psychological arousal associated with activities requiring different patterns of cognitive/affective and interpersonal involvement (activity engagement). We demonstrated that these signals can differentiate among patterns of arousal associated with these activities with high accuracy (two levels: 81%, three levels: 74%). These results demonstrate the potential for development of a real-time, motor- and language-free measure for describing the experiences of children and youth with disabilities.

Design and evaluation of a novel microphone-based mechanomyography sensor with cylindrical and conical acoustic chambers

Mechanomyography has recently been proposed as a control modality for alternative access technologies for individuals with disabilities. However, MMG recordings are highly susceptible to contamination from limb movements. Pressure-based transducers are touted to be the most robust to external movement although there is some debate about their optimal chamber geometry, in terms of low frequency gain and spectral flatness. To investigate the question of preferred geometry, transducers with cylindrical and conical chambers of varying dimensions were designed, manufactured and tested. Using a computer-controlled electrodynamic shaker, the frequency response of each chamber geometry was empirically derived. Of the cylindrical chambers, the highest gain and the flattest frequency response was exhibited by a chamber 10 mm in diameter and 5-7 mm in height. However, conical chambers offered an average rise in gain of 6.79 ± 1.06 dB/Hz over that achievable with cylindrical geometries. The highest gain and flattest response was achieved with a transducer consisting of a low-frequency MEMS microphone, a 4 μm aluminized mylar membrane and a rigid conical chamber 7 mm in diameter and 5mm in height. This design is recommended for MMG applications where limb movement is prevalent.

A brain-computer interface based on bilateral transcranial Doppler ultrasound

In this study, we investigate the feasibility of a BCI based on transcranial Doppler ultrasound (TCD), a medical imaging technique used to monitor cerebral blood flow velocity. We classified the cerebral blood flow velocity changes associated with two mental tasks - a word generation task, and a mental rotation task. Cerebral blood flow velocity was measured simultaneously within the left and right middle cerebral arteries while nine able-bodied adults alternated between mental activity (i.e. word generation or mental rotation) and relaxation. Using linear discriminant analysis and a set of time-domain features, word generation and mental rotation were classified with respective average accuracies of 82.9%10.5 and 85.7%10.0 across all participants. Accuracies for all participants significantly exceeded chance. These results indicate that TCD is a promising measurement modality for BCI research.