Perceptually Equivalent Judgments Made Visually and via Haptic Sensory-Substitution Devices

A direct comparison of sound and vibration as sources of stimulation for a sensory substitution glove

Sensory substitution devices (SSDs) facilitate the detection of environmental information through enhancement of touch and/or hearing capabilities. Research has demonstrated that several tasks can be successfully completed using acoustic, vibrotactile, and multimodal devices. The suitability of a substituting modality is also mediated by the type of information required to perform the specific task. The present study tested the adequacy of touch and hearing in a grasping task by utilizing a sensory substitution glove. The substituting modalities inform, through increases in stimulation intensity, about the distance between the fingers and the objects. A psychophysical experiment of magnitude estimation was conducted. Forty blindfolded sighted participants discriminated equivalently the intensity of both vibrotactile and acoustic stimulation, although they experienced some difficulty with the more intense stimuli. Additionally, a grasping task involving cylindrical objects of varying diameters, distances and orientations was performed. Thirty blindfolded sighted participants were divided into vibration, sound, or multimodal groups. High performance was achieved (84% correct grasps) with equivalent success rate between groups. Movement variables showed more precision and confidence in the multimodal condition. Through a questionnaire, the multimodal group indicated their preference for using a multimodal SSD in daily life and identified vibration as their primary source of stimulation. These results demonstrate that there is an improvement in performance with specific-purpose SSDs, when the necessary information for a task is identified and coupled with the delivered stimulation. Furthermore, the results suggest that it is possible to achieve functional equivalence between substituting modalities when these previous steps are met.

Perceiving multiple properties of a single person-probe-surface system

We performed three experiments to investigate the extent to which people can selectively perceive different properties of a person-probe-surface system, and the extent to which such abilities are supported by sensitivities to different (components of) invariant mechanical parameters. Participants probed a surface with a wielded object and attempted to perceive an exteroceptive property of the probe (whole length), an exproprioceptive property of the probe (partial length), and an exteroceptive property of the surface (distance). After controlling for potential confounds, we found that participants were able to selectively perceive each of these properties, and that in each case, such abilities were supported by sensitivity to distinct components of mechanical invariants. The results suggest that people can selectively attend to properties both across and within components of a person-probe-surface system and are discussed in the context of the invariant mechanical parameters that support this ability.

Empirical Evidence for Extended Cognitive Systems

We present an empirically supported theoretical and methodological framework for quantifying the system‐level properties of person‐plus‐tool interactions in order to answer the question: “Are person‐plus‐tool‐systems extended cognitive systems?” Nineteen participants provided perceptual judgments regarding their ability to pass through apertures of various widths while using visual information, blindfolded wielding a rod, or blindfolded wielding an Enactive Torch—a vibrotactile sensory‐substitution device for detecting distance. Monofractal, multifractal, and recurrence quantification analyses were conducted to assess features of person‐plus‐tool movement dynamics. Trials where people utilized the rod or Enactive Torch demonstrated stable “self‐similarity,” or indices of healthy and adaptive single systems, regardless of aperture width, trial order, features of the participants’ judgments, and participant characteristics. Enactive Torch trials exhibited a somewhat greater range of dynamic fluctuations than the rod trials, as well as less movement recurrence, suggesting that the Enactive Torch allowed for more exploratory movements. Findings provide support for the notion that person‐plus‐tool systems can be classified as extended cognitive systems and a framework for quantifying system‐level properties of these systems. Implications concerning future research on extended cognition are discussed.

A self-training program for sensory substitution devices

Sensory Substitution Devices (SSDs) convey visual information through audition or touch, targeting blind and visually impaired individuals. One bottleneck towards adopting SSDs in everyday life by blind users, is the constant dependency on sighted instructors throughout the learning process. Here, we present a proof-of-concept for the efficacy of an online self-training program developed for learning the basics of the EyeMusic visual-to-auditory SSD tested on sighted blindfolded participants. Additionally, aiming to identify the best training strategy to be later re-adapted for the blind, we compared multisensory vs. unisensory as well as perceptual vs. descriptive feedback approaches. To these aims, sighted participants performed identical SSD-stimuli identification tests before and after ~75 minutes of self-training on the EyeMusic algorithm. Participants were divided into five groups, differing by the feedback delivered during training: auditory-descriptive, audio-visual textual description, audio-visual perceptual simultaneous and interleaved, and a control group which had no training. At baseline, before any EyeMusic training, participants SSD objects’ identification was significantly above chance, highlighting the algorithm’s intuitiveness. Furthermore, self-training led to a significant improvement in accuracy between pre- and post-training tests in each of the four feedback groups versus control, though no significant difference emerged among those groups. Nonetheless, significant correlations between individual post-training success rates and various learning measures acquired during training, suggest a trend for an advantage of multisensory vs. unisensory feedback strategies, while no trend emerged for perceptual vs. descriptive strategies. The success at baseline strengthens the conclusion that cross-modal correspondences facilitate learning, given SSD algorithms are based on such correspondences. Additionally, and crucially, the results highlight the feasibility of self-training for the first stages of SSD learning, and suggest that for these initial stages, unisensory training, easily implemented also for blind and visually impaired individuals, may suffice. Together, these findings will potentially boost the use of SSDs for rehabilitation.

Where Is the Action in Perception? An Exploratory Study With a Haptic Sensory Substitution Device

Enactive cognitive science (ECS) and ecological psychology (EP) agree that active movement is important for perception, but they remain ambiguous regarding the precise role of agency. EP has focused on the notion of sensorimotor invariants, according to which bodily movements play an instrumental role in perception. ECS has focused on the notion of sensorimotor contingencies, which goes beyond an instrumental role because skillfully regulated movements are claimed to play a constitutive role. We refer to these two hypotheses as instrumental agency and constitutive agency, respectively. Evidence comes from a variety of fields, including neural, behavioral, and phenomenological research, but so far with confounds that prevent an experimental distinction between these hypotheses. Here we advance the debate by proposing a novel double-participant setup that aims to isolate agency as the key variable that distinguishes bodily movement in active and passive conditions of perception. We pilot this setup with a psychological study of width discrimination using the Enactive Torch, a haptic sensory substitution device. There was no evidence favoring the stronger hypothesis of constitutive agency over instrumental agency. However, we caution that during debriefing several participants reported using cognitive strategies that did not rely on spatial perception. We conclude that this approach is a viable direction for future research, but that greater care is required to establish and confirm the desired modality of first-person experience.

Distributed cognition criteria: Defined, operationalized, and applied to human-dog systems

Distributed cognition generally refers to situations in which task requirements are shared among multiple agents or, potentially, off-loaded onto the environment. With few exceptions, socially distributed cognition has largely been discussed in terms of intraspecific interactions. This conception fails to capture some forms of group-level cognition among human and non-human animals that are not readily measured or explained in mentalistic or verbal terms. In response to these limitations, we argue for a more stringent set of empirically-verifiable criteria for assessing whether a system is an instance of distributed cognition: interaction-dominant dynamics, agency, and shared task orientation. We apply this framework to humans and working dogs, and contrast the human-dog socially distributed cognitive system with humans using non-biological tools and human interaction with draft animals. The human-dog system illustrates three operationalizable factors for classifying phenomena as socially distributed cognition and extends the framework to interspecies distributed cognition.