Design Guidelines and Recommendations for Multimodal, Touchscreen-based Graphics

A direct comparison of sound and vibration as sources of stimulation for a sensory substitution glove

Sensory substitution devices (SSDs) facilitate the detection of environmental information through enhancement of touch and/or hearing capabilities. Research has demonstrated that several tasks can be successfully completed using acoustic, vibrotactile, and multimodal devices. The suitability of a substituting modality is also mediated by the type of information required to perform the specific task. The present study tested the adequacy of touch and hearing in a grasping task by utilizing a sensory substitution glove. The substituting modalities inform, through increases in stimulation intensity, about the distance between the fingers and the objects. A psychophysical experiment of magnitude estimation was conducted. Forty blindfolded sighted participants discriminated equivalently the intensity of both vibrotactile and acoustic stimulation, although they experienced some difficulty with the more intense stimuli. Additionally, a grasping task involving cylindrical objects of varying diameters, distances and orientations was performed. Thirty blindfolded sighted participants were divided into vibration, sound, or multimodal groups. High performance was achieved (84% correct grasps) with equivalent success rate between groups. Movement variables showed more precision and confidence in the multimodal condition. Through a questionnaire, the multimodal group indicated their preference for using a multimodal SSD in daily life and identified vibration as their primary source of stimulation. These results demonstrate that there is an improvement in performance with specific-purpose SSDs, when the necessary information for a task is identified and coupled with the delivered stimulation. Furthermore, the results suggest that it is possible to achieve functional equivalence between substituting modalities when these previous steps are met.

Accessible Tutoring Platform Using Audio-Tactile Graphics Adapted for Visually Impaired People

One of the problems faced by people with blindness is access to materials presented in graphical form. There are many alternative forms of providing such information, but they are very often ineffective or have certain limitations. The development of mobile devices and touch sensors enabled the development of new tools to support such people. This study presents a solution called an accessible tutoring platform, using audio-tactile graphics for people with blindness. We aimed to research the influence of the developed platform for the alternative presentation of graphics information on better memorizing, recognizing, and learning. Another goal of the research was to verify the effectiveness of the proposed method for the alternative presentation of audio-tactile graphics. The effectiveness of the proposed solution was verified quantitatively and qualitatively on two groups of blind students from primary and secondary schools with the use of a developed platform and prepared materials for learning mathematics. The obtained research results show that the proposed method of verifying students' knowledge and auto-selecting exercises with adapted audio description positively influences the improvement of learning effectiveness.

The Accessibility of Data Visualizations on the Web for Screen Reader Users: Practices and Experiences During COVID-19

Data visualization has become an increasingly important means of effective data communication and has played a vital role in broadcasting the progression of COVID-19. Accessible data representations, on the other hand, have lagged behind, leaving areas of information out of reach for many blind and visually impaired (BVI) users. In this work, we sought to understand (1) the accessibility of current implementations of visualizations on the web; (2) BVI users’ preferences and current experiences when accessing data-driven media; (3) how accessible data representations on the web address these users’ access needs and help them navigate, interpret, and gain insights from the data; and (4) the practical challenges that limit BVI users’ access and use of data representations. To answer these questions, we conducted a mixed-methods study consisting of an accessibility audit of 87 data visualizations on the web to identify accessibility issues, an online survey of 127 screen reader users to understand lived experiences and preferences, and a remote contextual inquiry with 12 of the survey respondents to observe how they navigate, interpret and gain insights from accessible data representations. Our observations during this critical period of time provide an understanding of the widespread accessibility issues encountered across online data visualizations, the impact that data accessibility inequities have on the BVI community, the ways screen reader users sought access to data-driven information and made use of online visualizations to form insights, and the pressing need to make larger strides towards improving data literacy, building confidence, and enriching methods of access. Based on our findings, we provide recommendations for researchers and practitioners to broaden data accessibility on the web.

Where's My Button? Evaluating the User Experience of Surface Haptics in Featureless Automotive User Interfaces

Advancements in user interface technologies and demands of design engineering led to increasing implementation of large and mostly flat interactive surfaces in automotive. Recent discussions in the context of in-vehicle usage of touchscreens advocate for the use of haptic feedback to restore the explore- and feel-qualities typically experienced in traditional physical button interfaces that contribute to intuitive, eyes-free, and tactually rich interactions. Haptic technologies that include a friction modulation approach seem especially promising to convey a high-quality feeling. This research reports an experience-oriented evaluation of an electrostatic friction haptic display in an in-vehicle direct touch interaction context. The evaluation was based on an automotive multitask setting (primary driving-task and secondary target-selection-task) with a 2 × 2 feedback modality design (factors haptic/audio with levels absent/present). The objective variables (response time, errors, and performance on the primary task) did not differ between feedback modalities. Any additional feedback to a visual baseline enhanced the user experience, with the multimodal feedback being preferred by most participants. Surface haptics was perceived as a novel yet unexpected type of haptic feedback. We discuss the implications for the haptic design of programmable friction displays and provide an initial set of guidelines for this innovative technology.

Comparing Map Learning between Touchscreen-Based Visual and Haptic Displays: A Behavioral Evaluation with Blind and Sighted Users

The ubiquity of multimodal smart devices affords new opportunities for eyes-free applications for conveying graphical information to both sighted and visually impaired users. Using previously established haptic design guidelines for generic rendering of graphical content on touchscreen interfaces, the current study evaluates the learning and mental representation of digital maps, representing a key real-world translational eyes-free application. Two experiments involving 12 blind participants and 16 sighted participants compared cognitive map development and test performance on a range of spatio-behavioral tasks across three information-matched learning-mode conditions: (1) our prototype vibro-audio map (VAM), (2) traditional hardcopy-tactile maps, and (3) visual maps. Results demonstrated that when perceptual parameters of the stimuli were matched between modalities during haptic and visual map learning, test performance was highly similar (functionally equivalent) between the learning modes and participant groups. These results suggest equivalent cognitive map formation between both blind and sighted users and between maps learned from different sensory inputs, providing compelling evidence supporting the development of amodal spatial representations in the brain. The practical implications of these results include empirical evidence supporting a growing interest in the efficacy of multisensory interfaces as a primary interaction style for people both with and without vision. Findings challenge the long-held assumption that blind people exhibit deficits on global spatial tasks compared to their sighted peers, with results also providing empirical support for the methodological use of sighted participants in studies pertaining to technologies primarily aimed at supporting blind users.

Image Accessibility for Screen Reader Users: A Systematic Review and a Road Map

A number of studies have been conducted to improve the accessibility of images using touchscreen devices for screen reader users. In this study, we conducted a systematic review of 33 papers to get a holistic understanding of existing approaches and to suggest a research road map given identified gaps. As a result, we identified types of images, visual information, input device and feedback modalities that were studied for improving image accessibility using touchscreen devices. Findings also revealed that there is little study how the generation of image-related information can be automated. Moreover, we confirmed that the involvement of screen reader users is mostly limited to evaluations, while input from target users during the design process is particularly important for the development of assistive technologies. Then we introduce two of our recent studies on the accessibility of artwork and comics, AccessArt and AccessComics, respectively. Based on the identified key challenges, we suggest a research agenda for improving image accessibility for screen reader users.