Effects of Virtual Reality-Based Multimodal Audio-Tactile Cueing in Patients With Spatial Attention Deficits: Pilot Usability Study

BACKGROUND

Virtual reality (VR) devices are increasingly being used in medicine and other areas for a broad spectrum of applications. One of the possible applications of VR involves the creation of an environment manipulated in a way that helps patients with disturbances in the spatial allocation of visual attention (so-called hemispatial neglect). One approach to ameliorate neglect is to apply cross-modal cues (ie, cues in sensory modalities other than the visual one, eg, auditory and tactile) to guide visual attention toward the neglected space. So far, no study has investigated the effects of audio-tactile cues in VR on the spatial deployment of visual attention in neglect patients.

OBJECTIVE

This pilot study aimed to investigate the feasibility and usability of multimodal (audio-tactile) cueing, as implemented in a 3D VR setting, in patients with neglect, and obtain preliminary results concerning the effects of different types of cues on visual attention allocation compared with noncued conditions.

METHODS

Patients were placed in a virtual environment using a head-mounted display (HMD). The inlay of the HMD was equipped to deliver tactile feedback to the forehead. The task was to find and flag appearing birds. The birds could appear at 4 different presentation angles (lateral and paracentral on the left and right sides), and with (auditory, tactile, or audio-tactile cue) or without (no cue) a spatially meaningful cue. The task usability and feasibility, and 2 simple in-task measures (performance and early orientation) were assessed in 12 right-hemispheric stroke patients with neglect (5 with and 7 without additional somatosensory impairment).

RESULTS

The new VR setup showed high usability (mean score 10.2, SD 1.85; maximum score 12) and no relevant side effects (mean score 0.833, SD 0.834; maximum score 21). A repeated measures ANOVA on task performance data, with presentation angle, cue type, and group as factors, revealed a significant main effect of cue type (F_30,3=9.863; P<.001) and a significant 3-way interaction (F_90,9=2.057; P=.04). Post-hoc analyses revealed that among patients without somatosensory impairment, any cue led to better performance compared with no cue, for targets on the left side, and audio-tactile cues did not seem to have additive effects. Among patients with somatosensory impairment, performance was better with both auditory and audio-tactile cueing than with no cue, at every presentation angle; conversely, tactile cueing alone had no significant effect at any presentation angle. Analysis of early orientation data showed that any type of cue triggered better orientation in both groups for lateral presentation angles, possibly reflecting an early alerting effect.

CONCLUSIONS

Overall, audio-tactile cueing seems to be a promising method to guide patient attention. For instance, in the future, it could be used as an add-on method that supports attentional orientation during established therapeutic approaches.

Development of a Search Task Using Immersive Virtual Reality: Proof-of-Concept Study

BACKGROUND

Serious games are gaining increasing importance in neurorehabilitation since they increase motivation and adherence to therapy, thereby potentially improving its outcome. The benefits of serious games, such as the possibility to implement adaptive feedback and the calculation of comparable performance measures, can be even further improved by using immersive virtual reality (iVR), allowing a more intuitive interaction with training devices and higher ecological validity.

OBJECTIVE

This study aimed to develop a visual search task embedded in a serious game setting for iVR, including self-adapting difficulty scaling, thus being able to adjust to the needs and ability levels of different groups of individuals.

METHODS

In a two-step process, a serious game in iVR (bird search task) was developed and tested in healthy young (n=21) and elderly (n=23) participants and in a group of patients with impaired visual exploration behavior (ie, patients with hemispatial neglect after right-hemispheric stroke; n=11). Usability, side effects, game experience, immersion, and presence of the iVR serious game were assessed by validated questionnaires. Moreover, in the group of stroke patients, the performance in the iVR serious game was also considered with respect to hemispatial neglect severity, as assessed by established objective hemispatial neglect measures.

RESULTS

In all 3 groups, reported usability of the iVR serious game was above 4.5 (on a Likert scale with scores ranging from 1 to 5) and reported side effects were infrequent and of low intensity (below 1.5 on a Likert scale with scores ranging from 1 to 4). All 3 groups equally judged the iVR serious game as highly motivating and entertaining. Performance in the game (in terms of mean search time) showed a lateralized increase in search time in patients with hemispatial neglect that varied strongly as a function of objective hemispatial neglect severity.

CONCLUSIONS

The developed iVR serious game, "bird search task," was a motivating, entertaining, and immersive task, which can, due to its adaptive difficulty scaling, adjust and be played by different populations with different levels of skills, including individuals with cognitive impairments. As a complementary finding, it seems that performance in the game is able to capture typical patterns of impaired visual exploration behavior in hemispatial neglect, as there is a high correlation between performance and neglect severity as assessed with a cancellation task.

Quantifying Human Avoidance Behavior in Immersive Virtual Reality

Avoidance behavior is a key symptom of most anxiety disorders and a central readout in animal research. However, the quantification of real-life avoidance behavior in humans is typically restricted to clinical populations, who show actual avoidance of phobic objects. In experimental approaches for healthy participants, many avoidance tasks utilize button responses or a joystick navigation on the screen as indicators of avoidance behavior. To allow the ecologically valid assessment of avoidance behavior in healthy participants, we developed a new automated immersive Virtual Reality paradigm, where participants could freely navigate in virtual 3-dimensional, 360-degrees scenes by real naturalistic body movements. A differential fear conditioning procedure was followed by three newly developed behavioral tasks to assess participants’ avoidance behavior of the conditioned stimuli: an approach, a forced-choice, and a search task. They varied in instructions, degrees of freedom, and high or low task-related relevance of the stimuli. We initially examined the tasks in a quasi-experiment (N = 55), with four consecutive runs and various experimental adaptations. Here, although we observed avoidance behavior in all three tasks after additional reinforcement, we only detected fear-conditioned avoidance behavior in the behavioral forced-choice and search tasks. These findings were largely replicated in a confirmatory experiment (N = 72) with randomized group allocation, except that fear-conditioned avoidance behavior was only manifest in the behavioral search task. This supports the notion that the behavioral search task is sensitive to detect avoidance behavior after fear conditioning only, whereas the behavioral approach and forced-choice tasks are still able to detect “strong” avoidance behavior after fear conditioning and additional reinforcement.

Use-inspired basic research in medical image perception

This journal is dedicated to "use-inspired basic research" where a problem in the world shapes the hypotheses for a study in the laboratory. This brief review presents several examples of "use-inspired basic research" in the area of medical image perception. These are cases where the field of radiology raises an interesting issue in visual cognition. Basic research on those issues may then lead to proposals to improve performance on clinical tasks in medical image perception. Of the six examples given here, the first three ask essentially perceptual questions: How can stereopsis improve medical image perception? How shall we assess the tradeoff between radiation dose and image quality? How does the choice of colors change the interpretation of medical images? The second three examples address attentional issues in those aspects of radiology that can be described as visual search problems: Can eye tracking help us understand errors in radiologic search? What happens if the number of targets in an image is unknown? What happens if, as in radiology screening programs, the target of search is very rare?

Through the Google Glass: The impact of heads-up displays on visual attention

In five experiments, we evaluated how secondary information presented on a heads-up display (HUD) impacts performance of a concurrent visual attention task. To do so, we had participants complete a primary visual search task under a variety of secondary load conditions (a single word presented on Google Glass during each search trial). Processing of secondary information was measured through a recognition memory task. Other manipulations included relevance (Experiments 1-4) and temporal onset of secondary information relative to the primary task (Experiment 3). Secondary information was always disruptive to the visual search, regardless of temporal onset and even when participants were instructed to ignore it. These patterns were evident in search tasks reflective of both selective (Experiments 1-3) and preattentive (Experiment 4) attentional mechanisms, and were not a result of onset-offset attentional capture (Experiment 5). Recognition memory for secondary information was always above chance. Our findings suggest that HUD-based visual information is profoundly disruptive to attentional processes and largely immune to user-centric prioritization.

Classification of Eye Fixation Related Potentials for Variable Stimulus Saliency

OBJECTIVE

Electroencephalography (EEG) and eye tracking can possibly provide information about which items displayed on the screen are relevant for a person. Exploiting this implicit information promises to enhance various software applications. The specific problem addressed by the present study is that items shown in real applications are typically diverse. Accordingly, the saliency of information, which allows to discriminate between relevant and irrelevant items, varies. As a consequence, recognition can happen in foveal or in peripheral vision, i.e., either before or after the saccade to the item. Accordingly, neural processes related to recognition are expected to occur with a variable latency with respect to the eye movements. The aim was to investigate if relevance estimation based on EEG and eye tracking data is possible despite of the aforementioned variability.

APPROACH

Sixteen subjects performed a search task where the target saliency was varied while the EEG was recorded and the unrestrained eye movements were tracked. Based on the acquired data, it was estimated which of the items displayed were targets and which were distractors in the search task.

RESULTS

Target prediction was possible also when the stimulus saliencies were mixed. Information contained in EEG and eye tracking data was found to be complementary and neural signals were captured despite of the unrestricted eye movements. The classification algorithm was able to cope with the experimentally induced variable timing of neural activity related to target recognition.

SIGNIFICANCE

It was demonstrated how EEG and eye tracking data can provide implicit information about the relevance of items on the screen for potential use in online applications.

Beyond the search barrier: A new task for assessing object individuation in young infants

Object individuation, the capacity to track the identity of objects when perceptual contact is lost and then regained, is fundamental to human cognition. A great deal of research using the violation-of-expectation method has been conducted to investigate the development of object individuation in infancy. At the same time, there is a growing need for converging methods of study. Reported here are data obtained with from a newly developed search task that can be used with infants as young as 5 months of age. The results suggest that this method is a sensitive measure of object individuation in young infants and demonstrate the advantages of using converging methods of study.