Effects of control button arrangements on human response to auditory and visual signals

Design of auditory P300-based brain-computer interfaces with a single auditory channel and no visual support

Non-invasive brain-computer interfaces (BCIs) based on an event-related potential (ERP) component, P300, elicited via the oddball paradigm, have been extensively developed to enable device control and communication. While most P300-based BCIs employ visual stimuli in the oddball paradigm, auditory P300-based BCIs also need to be developed for users with unreliable gaze control or limited visual processing. Specifically, auditory BCIs without additional visual support or multi-channel sound sources can broaden the application areas of BCIs. This study aimed to design optimal stimuli for auditory BCIs among artificial (e.g., beep) and natural (e.g., human voice and animal sounds) sounds in such circumstances. In addition, it aimed to investigate differences between auditory and visual stimulations for online P300-based BCIs. As a result, natural sounds led to both higher online BCI performance and larger differences in ERP amplitudes between the target and non-target compared to artificial sounds. However, no single type of sound offered the best performance for all subjects; rather, each subject indicated different preferences between the human voice and animal sound. In line with previous reports, visual stimuli yielded higher BCI performance (average 77.56%) than auditory counterparts (average 54.67%). In addition, spatiotemporal patterns of the differences in ERP amplitudes between target and non-target were more dynamic with visual stimuli than with auditory stimuli. The results suggest that selecting a natural auditory stimulus optimal for individual users as well as making differences in ERP amplitudes between target and non-target stimuli more dynamic may further improve auditory P300-based BCIs.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11571-022-09901-3.

Visual presentation modality's superiority in the detection of concealed information: A comparison of the efficiencies of the P300-based Complex Trial Protocol in visual versus auditory modalities

Some studies have shown a superiority of visual vs. auditory item presentation in the Complex Trial Protocol (CTP), which is a countermeasure-resistant version of the P300-based Concealed Information Test (CIT). But those studies used elaborately- rehearsed autobiographical information as stimuli, instead of incidentally-acquired crime-related information. Thus, the relative superiority of the visual as opposed to the auditory modality in detecting episodic crime-related information is still unknown. The present study also improved on the usual mock crime scenario by adding a mock disposal task between a mock theft and administration of a CTP test to increase stimulus saliency. In this CTP, the probe and the irrelevant items were presented visually or acoustically on alternating trials, while target and non-target stimuli were simultaneously presented in visual and auditory modalities. The results showed that the P300 amplitude differences of probe minus irrelevant items presented in the visual modality were significantly larger compared to the auditory modality, and the detection rate of the guilty participants was also significantly higher for the visual (14/16) versus auditory modality (5/16). These results suggest a superiority of visual vs. auditory presentation when a CTP is used to detect crime-related information in a mock crime scenario.

Effect of Visual-Verbal Load and Spatial Compatibility on Stimulus Response

This study examined the effects of visual-verbal load (as measured by a visually presented reading-memory task with three levels) on a visual/auditory stimulus-response task. The three levels of load were defined as follows: “No Load” meant no other stimuli were presented concurrently; “Free Load” meant that a letter (A, B, C, or D) appeared at the same time as the visual or auditory stimulus; and “Force Load” was the same as “Free Load,” but the participants were also instructed to count how many times the letter A appeared. The stimulus-response task also had three levels: “irrelevant,” “compatible,” and “incompatible” spatial conditions. These required different key-pressing responses. The visual stimulus was a red ball presented either to the left or to the right of the display screen, and the auditory stimulus was a tone delivered from a position similar to that of the visual stimulus. Participants also processed an irrelevant stimulus. The results indicated that participants perceived auditory stimuli earlier than visual stimuli and reacted faster under stimulus-response compatible conditions. These results held even under a high visual-verbal load. These findings suggest the following guidelines for systems used in driving: an auditory source, appropriately compatible signal and manual-response positions, and a visually simplified background.

Comparing the Effects of Light Alcohol Consumption on Human Response to Auditory and Visual Stimuli

The purpose of this study is to examine the effects of various levels of alcohol consumption on human response to auditory and visual stimuli in terms of reaction time, movement time, total reaction time, and error rate. Placebo level and three low-level alcohol doses were randomly assigned to 20 male university student volunteers. 30 min. after consuming the alcohol or placebo, participants responded to either auditory or visual stimuli. Total reaction time increased significantly at the mid-low dose of alcohol (0.3 g/kg). For alcohol doses less than .5 g/kg, the change in total reaction time was confined to reaction time, i.e., the processing time between onset of stimulus and onset of movement. Effects of alcohol were significantly more pronounced in the choice-type tests. Notably, the effects of alcohol on total reaction time and error rate were significant for auditory but not visual stimuli.