Therapeutic potential of robots for people who stutter: a preliminary study

Introduction

Growing anecdotal evidence suggests the feasibility of robotic intervention for people who suffer from disorders related to state anxiety. Few studies have been conducted on utilizing robots for persons who stutter (PWS). The present study examines the feasibility of using a robot for speech therapy for PWS.

Methods

We prepared four settings (i.e., interviews with unfamiliar persons, interviews with unfamiliar communication robots, reading sentences aloud with a tandem robot that can utter the same words as a user by repeating the user's voice after a short delay, and reading sentences aloud while being alone). We assessed the potential of the robots as both interlocutors and practice partners in training with delayed auditory feedback (DAF) for PWS. Moreover, we assessed the relationship between the trait of stuttering and the participants' affinity to the robots.

Results

Eleven PWS participated in the study. Eight (72.7%) participants had fewer stuttering-related psychological symptoms when they communicated with robots than when they communicated with humans. Spearman's rank correlation analysis revealed that there was a significant negative correlation between the Modified Erickson Communication Attitude scale (S-24) and the difference between the scores for stuttering-related psychological symptoms pertaining to the communication robot and humans (p < 0.01). Six participants (54.5%) had fewer stuttering-related psychological symptoms when they read aloud with the tandem robot than when they read aloud alone. There were significant positive correlations between S-24 and the differences between the scores for stuttering-related psychological symptoms when reading aloud with the tandem robot and those when reading aloud alone (p < 0.01).

Discussion

The communication robot and tandem utterance robot can sometimes be burdensome, although both robots were always easier to talk to for PWS in this preliminary study. The participants with positive speech-related attitudes were more inclined to decrease stuttering-related psychological symptoms when communicating with CommU than when communicating with humans. The participants whose speech-related attitudes were negative were more inclined to show a decrease in stuttering-related psychological symptoms when reading aloud with the tandem robot. Further studies are needed to provide more detailed information.

Audiomotor Temporal Recalibration Modulates Decision Criterion of Self-Agency but Not Perceptual Sensitivity

Exposure to delayed sensory feedback changes perceived simultaneity between action and feedback [temporal recalibration (TR)] and even modulates the sense of agency (SoA) over the feedback. To date, however, it is not clear whether the modulation of SoA by TR is caused by a change in perceptual sensitivity or decision criterion of self-agency. This experimental research aimed to tease apart these two by applying the signal detection theory (SDT) to the agency judgment over auditory feedback after voluntary action. Participants heard a short sequence of tone pips with equal inter-onset intervals, and they reproduced it by pressing a computer mouse. The delay of each tone pip after the mouse press was manipulated as 80 (baseline) or 180 ms (delayed). Subsequently, the participants reproduced it, in which the delay was fixed at 80 ms and there was a 50% chance that the computer took over the control of the tone pips from the participants. The participants’ task was to discriminate who controlled the tone pips and to judge synchrony between tone pips and mouse presses. Results showed that the modulation of the SoA by the TR is caused by a shift in the decision criterion but not in the perceptual sensitivity of agency.

Too Late! Influence of Temporal Delay on the Neural Processing of One's Own Incidental and Intentional Action-Induced Sounds

The influence of delayed auditory feedback on action evaluation and execution of real-life action-induced sounds apart from language and music is still poorly understood. Here, we examined how a temporal delay impacted the behavioral evaluation and neural representation of hurdling and tap-dancing actions in a functional magnetic resonance imaging (fMRI) experiment, postulating that effects of delay diverge between the two, as we create action-induced sounds intentionally in tap dancing, but incidentally in hurdling. Based on previous findings, we expected that conditions differ regarding the engagement of the supplementary motor area (SMA), posterior superior temporal gyrus (pSTG), and primary auditory cortex (A1). Participants were videotaped during a 9-week training of hurdling and tap dancing; in the fMRI scanner, they were presented with point-light videos of their own training videos, including the original or the slightly delayed sound, and had to evaluate how well they performed on each single trial. For the undelayed conditions, we replicated A1 attenuation and enhanced pSTG and SMA engagement for tap dancing (intentionally generated sounds) vs. hurdling (incidentally generated sounds). Delayed auditory feedback did not negatively influence behavioral rating scores in general. Blood-oxygen-level-dependent (BOLD) response transiently increased and then adapted to repeated presentation of point-light videos with delayed sound in pSTG. This region also showed a significantly stronger correlation with the SMA under delayed feedback. Notably, SMA activation increased more for delayed feedback in the tap-dancing condition, covarying with higher rating scores. Findings suggest that action evaluation is more strongly based on top–down predictions from SMA when sounds of intentional action are distorted.

Speech-Induced Suppression for Delayed Auditory Feedback in Adults Who Do and Do Not Stutter

Speech-induced suppression is the normal, relative amplitude reduction of the auditory evoked potential for self-, compared to externally-generated, auditory stimulation. It remains controversial as to whether adults who stutter exhibit expected auditory modulation during speech; some studies have reported a significant difference between stuttering and fluent groups in speech-induced suppression during speech movement planning, while others have not. We compared auditory evoked potentials (N1 component) for auditory feedback arising from one’s own voice (Speaking condition) with passive listening to a recording of one’s own voice (Listening condition) in 24 normally-fluent speakers and 16 adults who stutter under various delayed auditory feedback (DAF) time conditions (100 ms, 200 ms, 500 ms, and 1,000 ms). We presented the participant’s own voice with a delay, immediately after presenting it without a delay. Our working hypothesis was that the shorter the delay time, the more likely the delayed sound is perceived as self-generated. Therefore, shorter delay time conditions are proposed to result in relatively enhanced suppression of the auditory system. Results showed that in fluent speakers, the shorter the delay time, the more the auditory evoked potential in the Speaking condition tended to be suppressed. In the Listening condition, there was a larger evoked potential with shorter delay times. As a result, speech-induced suppression was only significant at the short delay time conditions of 100 and 200 ms. Adults who stutter did not show the opposing changes in the Speaking and Listening conditions seen in the fluent group. Although the evoked potential in the Listening condition tended to decrease as the delay time increased, that in the Speaking condition did not show a distinct trend, and there was a significant suppression only at 200 ms delay. For the 200 ms delay condition, speakers with more severe stuttering showed significantly greater speech-induced suppression than those with less severe stuttering. This preliminary study suggests our methods for investigating evoked potentials by presenting own voice with a delay may provide a clue as to the nature of auditory modulation in stuttering.

Delayed Auditory Feedback and Transcranial Direct Current Stimulation Treatment for the Enhancement of Speech Fluency in Adults Who Stutter: Protocol for a Randomized Controlled Trial

Background

Stuttering is a complex speech disorder that affects speech fluency. Recently, it has been shown that noninvasive brain stimulation may be useful to enhance the results of fluency interventions in adults who stutter. Delayed auditory feedback (DAF) is a method to enhance speech fluency in individuals who stutter. Adjunctive interventions are warranted to enhance the efficacy of this intervention.

Objective

Individuals who stutter have pathological activation patterns in the primary and secondary auditory areas. Consequently, in this study, we hypothesize that stimulation of these areas might be promising as an adjunctive method to fluency training via DAF to enhance speech therapy success in individuals with a stutter. We will systematically test this hypothesis in this study.

Methods

This study is designed as a randomized, double-blind, sham-controlled clinical trial. All participants will receive DAF. The intervention group will additionally receive real transcranial direct current stimulation, while the control group will be exposed to sham stimulation. The assignment of the participants to one of these groups will be randomized. Before starting the treatment program, 2 preintervention assessments will be conducted to determine the severity of stuttering. Once these assessments are completed, each subject will participate in 6 intervention sessions. Postintervention assessments will be carried out immediately and 1 week after the last intervention session. Subsequently, to explore the long-term stability of the treatment results, the outcome parameters will be obtained in follow-up assessments 6 weeks after the treatment. The primary outcome measurement—the percentage of stuttered syllables—will be calculated in pre-, post-, and follow-up assessments; the secondary outcomes will be the scores of the following questionnaires: the Stuttering Severity Instrument–Fourth Edition and the Overall Assessment of the Speaker’s Experience of Stuttering.

Results

This protocol was funded in 2019 and approved by the Research Ethics Committee of the Iran University of Medical Sciences in June 2019. Data collection started in October 2019. As of February 2020, we have enrolled 30 participants. We expect data analysis to be completed in April 2020, and results will be published in summer 2020.

Conclusions

We anticipate that this study will show an adjunctive effect of transcranial direct current stimulation, when combined with DAF, on stuttering. This should include not only a reduction in the percentage of stuttered syllables but also improved physical behavior and quality of life in adults who stutter.

Trial Registration

ClinicalTrial.gov NCT03990168; https://clinicaltrials.gov/ct2/show/NCT03990168

International Registered Report Identifier (IRRID)

DERR1-10.2196/16646

The effect of delayed auditory feedback (DAF) and frequency altered feedback (FAF) on speech production: cochlear implanted versus normal hearing individuals

Normal auditory feedback contributes to moment-to-moment control of speech production. Effects of auditory feedback's absence on hearing-impaired individuals are widely documented, but auditory perturbation has not been investigated. Our objective was to evaluate the effect of delayed auditory feedback (DAF) and frequency altered feedback (FAF) on speech production among prelingual cochlear implant (CI) users and normal hearing (NH) individuals, to evaluate CI users' reliance on auditory feedback. Twenty young adults (10 CI, 10 NH), without developmental and cognitive impairments, participated in the study. Under variable auditory feedback conditions, speech production (spontaneous or reading aloud) was measured using speech rate, percentage of interruptions, fundamental frequency (F0), and relative intensity. Results showed that (1) both DAF and FAF caused slower speech rates and more interruptions while reading aloud, with DAF having larger effect; (2) altered feedback produced no differences between groups, except an increase in F0 for CI users during DAF; and (3) CI users' ability to understand speech via phone and without lip-reading was positively correlated with performance under DAF. These findings suggest that auditory perturbation similarly affects speech production among prelingual CI users and NH individuals, indicating CI users depend on auditory feedback to the same degree as normal hearing individuals.

Exploring Different Types of Inhibition During Bilingual Language Production

Multilinguals have to control their languages constantly to produce accurate verbal output. They have to inhibit possible lexical competitors not only from the target language, but also from non-target languages. Bilinguals' training in inhibiting incongruent or irrelevant information has been used to endorse the so-called bilingual advantage in executive functions, assuming a transfer effect from language inhibition to domain-general inhibitory skills. Recent studies have suggested that language control may rely on language-specific inhibitory control mechanisms. In the present study, unbalanced highly proficient bilinguals completed a rapid naming multi-inhibitory task in two languages. The task assessed three types of inhibitory processes: inhibition of the non-target language, inhibition of lexical competitors, and inhibition of erroneous auditory feedback. The results showed an interaction between lexical competition and erroneous auditory feedback, but no interactions with the inhibition of the non-target language. The results suggested that different subcomponents of language inhibition are involved during bilingual language production.

Sensitivity of Speech Output to Delayed Auditory Feedback in Primary Progressive Aphasias

Delayed auditory feedback (DAF) is a classical paradigm for probing sensori-motor interactions in speech output and has been studied in various disorders associated with speech dysfluency and aphasia. However, little information is available concerning the effects of DAF on degenerating language networks in primary progressive aphasia: the paradigmatic "language-led dementias." Here we studied two forms of speech output (reading aloud and propositional speech) under natural listening conditions (no feedback delay) and under DAF at 200 ms, in a cohort of 19 patients representing all major primary progressive aphasia syndromes vs. healthy older individuals and patients with other canonical dementia syndromes (typical Alzheimer's disease and behavioral variant frontotemporal dementia). Healthy controls and most syndromic groups showed a quantitatively or qualitatively similar profile of reduced speech output rate and increased speech error rate under DAF relative to natural auditory feedback. However, there was no group effect on propositional speech output rate under DAF in patients with nonfluent primary progressive aphasia and logopenic aphasia. Importantly, there was considerable individual variation in DAF sensitivity within syndromic groups and some patients in each group (though no healthy controls) apparently benefited from DAF, showing paradoxically increased speech output rate and/or reduced speech error rate under DAF. This work suggests that DAF may be an informative probe of pathophysiological mechanisms underpinning primary progressive aphasia: identification of "DAF responders" may open up an avenue to novel therapeutic applications.

Neural Basis of the Time Window for Subjective Motor-Auditory Integration

Temporal contiguity between an action and corresponding auditory feedback is crucial to the perception of self-generated sound. However, the neural mechanisms underlying motor–auditory temporal integration are unclear. Here, we conducted four experiments with an oddball paradigm to examine the specific event-related potentials (ERPs) elicited by delayed auditory feedback for a self-generated action. The first experiment confirmed that a pitch-deviant auditory stimulus elicits mismatch negativity (MMN) and P300, both when it is generated passively and by the participant’s action. In our second and third experiments, we investigated the ERP components elicited by delayed auditory feedback for a self-generated action. We found that delayed auditory feedback elicited an enhancement of P2 (enhanced-P2) and a N300 component, which were apparently different from the MMN and P300 components observed in the first experiment. We further investigated the sensitivity of the enhanced-P2 and N300 to delay length in our fourth experiment. Strikingly, the amplitude of the N300 increased as a function of the delay length. Additionally, the N300 amplitude was significantly correlated with the conscious detection of the delay (the 50% detection point was around 200 ms), and hence reduction in the feeling of authorship of the sound (the sense of agency). In contrast, the enhanced-P2 was most prominent in short-delay (≤200 ms) conditions and diminished in long-delay conditions. Our results suggest that different neural mechanisms are employed for the processing of temporally deviant and pitch-deviant auditory feedback. Additionally, the temporal window for subjective motor–auditory integration is likely about 200 ms, as indicated by these auditory ERP components.

Atypical delayed auditory feedback effect and Lombard effect on speech production in high-functioning adults with autism spectrum disorder

Individuals with autism spectrum disorder (ASD) show impaired social interaction and communication, which may be related to their difficulties in speech production. To investigate the mechanisms of atypical speech production in this population, we examined feedback control by delaying the auditory feedback of their own speech, which degraded speech fluency. We also examined feedforward control by adding loud pink noise to the auditory feedback, which led to increased vocal effort in producing speech. The results of Japanese speakers show that, compared with neurotypical (NT) individuals, high-functioning adults with ASD (including Asperger’s disorder, autistic disorder, and pervasive developmental disorder not otherwise specified) were more affected by delayed auditory feedback but less affected by external noise. These findings indicate that, in contrast to NT individuals, those with ASD relied more on feedback control than on feedforward control in speech production, which is consistent with the hypothesis that this population exhibits attenuated Bayesian priors.

Concurrent sensorimotor temporal recalibration to different lags for the left and right hand

Perception of temporal synchrony between one’s own action and the sensory feedback of that action is quite flexible. We examined whether sensorimotor temporal recalibration (TR) involves central or motor-specific components by concurrently exposing the left and right hands to different lags. The experiment was composed of a pre-test, an adaptation phase, and a post-test. During the adaptation phase, participants tapped their left and right index fingers in alternating fashion while each tap induced an auditory feedback signal (a short click sound). One hand was exposed to a long delay between the tap and the sound (~150 ms), while the other hand was exposed to a subjective no-delay (~50 ms). Before and after the adaptation phase (the pre- and post-test), participants tried to tap in synchrony with pacer tones (ISI = 1000 ms). The results showed that the hand that was exposed to the delayed sound corrected for this delay by tapping earlier (a larger anticipation error) than the no-delay hand, indicating TR. Different amounts of TR were found when the left and right hand were concurrently exposed to the same versus different delays. With different exposure- delays for the two hands, there was a TR even for the hand that did not experience any delay in the feedback signal. However, it is not the case with the same exposure delay for the two hands. TR of the hand that experienced delayed feedback also occurred faster and was more complete (~40% greater than that of the hand with no subjective delay) if the two hands were exposed to the same rather than different delays (~20% greater than that of the hand with no subjective delay). These results suggest the existence of cross-talk between the hands, where both central and motor-specific components might be involved.

Individual variability in delayed auditory feedback effects on speech fluency and rate in normally fluent adults

PURPOSE

Delayed auditory feedback (DAF) is known to induce stuttering-like disfluencies (SLDs) and cause speech rate reductions in normally fluent adults, but the reason for speech disruptions is not fully known, and individual variation has not been well characterized. Studying individual variation in susceptibility to DAF may identify factors that predispose an individual to be more or less dependent on auditory feedback.

METHOD

Participants were 62 normally fluent adults. Each participant performed a spontaneous speech task in 250-ms DAF and amplified nondelayed auditory feedback (NAF) conditions. SLDs, other disfluencies (ODs), speech errors (SEs), and articulation rate (AR) were measured under each condition.

RESULTS

In the DAF condition, SLDs and SEs significantly increased, and AR decreased. Sex had a limited effect in that men exhibited higher rates of ODs and faster AR than women. More important, parametric cluster analysis identified that 2- and 3-subgroup solutions reveal important variation that differentiates tendencies toward disfluency changes and rate reduction under DAF, which are theoretically and empirically preferred to a single-group solution.

CONCLUSION

Individual variability in response to DAF may be accounted for by subgroups of individuals. This suggests that certain normally fluent individuals could be more dependent on intact feedback to maintain fluency.

The Build-Up and Transfer of Sensorimotor Temporal Recalibration Measured via a Synchronization Task

The timing relation between a motor action and the sensory consequences of that action can be adapted by exposing participants to artificially delayed feedback (temporal recalibration). Here, we demonstrate that a sensorimotor synchronization task (i.e., tapping the index finger in synchrony with a pacing signal) can be used as a measure of temporal recalibration. Participants were first exposed to a constant delay (~150 ms) between a voluntary action (a finger tap) and an external feedback stimulus of that action (a visual flash or auditory tone). A subjective "no-delay" condition (~50 ms) served as baseline. After a short exposure phase to delayed feedback participants performed the tapping task in which they tapped their finger in synchrony with a flash or tone. Temporal recalibration manifested itself in that taps were given ~20 ms earlier after exposure to 150 ms delays than in the case of 50 ms delays. This effect quickly built up (within 60 taps) and was bigger for auditory than visual adapters. In Experiment 2, we tested whether temporal recalibration would transfer across modalities by switching the modality of the adapter and pacing signal. Temporal recalibration transferred from visual adapter to auditory test, but not from auditory adapter to visual test. This asymmetric transfer suggests that sensory-specific effects are at play.

Audio-vocal monitoring system revealed by mu-rhythm activity

Understanding the neural mechanisms underlying speech production has a number of potential practical applications. Speech production involves multiple feedback loops. An audio-vocal monitoring system plays an important role in speech production, based on auditory feedback about the speaker’s own voice. Here we investigated the mu-rhythm activity associated with speech production by examining event-related desynchronization and synchronization in conditions of delayed auditory feedback (DAF) and noise feedback (Lombard). In Experiment 1, we confirmed that the mu-rhythms were detectable for a conventional finger-tapping task, and vocalization. In Experiment 2, we examined the mu-rhythms for imagined speech production. We tested whether the same motor-related mu-rhythm activity was exhibited while participants listened to their own voice, and while reading. The mu-rhythms were observed for overt vocalization and covert reading, while listening to simulated auditory feedback of the participants’ own voice reading text. In addition, we found that the mu-rhythm associated with listening was boosted and attenuated under the DAF and Lombard conditions, respectively. This is consistent with the notion that auditory feedback is important for the audio-vocal monitoring system in speech production. This paradigm may help clarify the way in which auditory feedback supports motor planning, as indexed by the motor-related mu-rhythm.

Effects of delayed visual feedback on grooved pegboard test performance

Using four experiments, this study investigates what amount of delay brings about maximal impairment under delayed visual feedback and whether a critical interval, such as that in audition, also exists in vision. The first experiment measured the Grooved Pegboard test performance as a function of visual feedback delays from 120 to 2120 ms in 16 steps. Performance sharply decreased until about 490 ms, then more gradually until 2120 ms, suggesting that two mechanisms were operating under delayed visual feedback. Since delayed visual feedback differs from delayed auditory feedback in that the former induces not only temporal but also spatial displacements between motor and sensory feedback, this difference could also exist in the mechanism responsible for spatial displacement. The second experiment was hence conducted to provide simultaneous haptic feedback together with delayed visual feedback to inform correct spatial position. The disruption was significantly ameliorated when information about spatial position was provided from a haptic source. The sharp decrease in performance of up to approximately 300 ms was followed by an almost flat performance. This is similar to the critical interval found in audition. Accordingly, the mechanism that caused the sharp decrease in performance in experiments 1 and 2 was probably mainly responsible for temporal disparity and is common across different modality–motor combinations, while the other mechanism that caused a rather gradual decrease in performance in experiment 1 was mainly responsible for spatial displacement. In experiments 3 and 4, the reliability of spatial information from the haptic source was reduced by wearing a glove or using a tool. When the reliability of spatial information was reduced, the data lay between those of experiments 1 and 2, and that a gradual decrease in performance partially reappeared. These results further support the notion that two mechanisms operate under delayed visual feedback.

Effects of delayed auditory feedback and frequency-shifted feedback on speech control and some potentials for future development of prosthetic aids for stammering

It has been known for at least a hundred years that the speech of a person who stammers becomes more fluent when alterations are made to the speaking environment. Alterations that lead to an improvement in fluency include a) noises that prevent a speaker hearing his or her own voice, and b) manipulations to the sound of a speaker's voice before it is heard. Examples of manipulations that have been made are introducing a delay, and shifting the voice up or down in frequency. The influences all these alterations have on fluent speakers and speakers who stammer, that have been established over the last century, are reviewed. In addition, the ways in which these phenomena have been explained for both fluent speaker and speakers who stammer are outlined. Several previous findings have potential significance for ways in which the fluency-enhancing effects of these alterations in speakers who stammer could be employed in clinical settings. These are highlighted and discussed, mainly in connection with the SpeechEasy prosthetic device for treating stammering.