The role of subvocalization in auditory imagery

Functional coupling between auditory memory and verbal transformations

The ability to parse sound mixtures into coherent auditory objects is fundamental to cognitive functions, such as speech comprehension and language acquisition. Yet, we still lack a clear understanding of how auditory objects are formed. To address this question, we studied a speech-specific case of perceptual multistability, called verbal transformations (VTs), in which a variety of verbal forms is induced by continuous repetition of a physically unchanging word. Here, we investigated the degree to which auditory memory through sensory adaptation influences VTs. Specifically, we hypothesized that when memory persistence is longer, participants are able to retain the current verbal form longer, resulting in sensory adaptation, which in turn, affects auditory perception. Participants performed VT and auditory memory tasks on different days. In the VT task, Japanese participants continuously reported their perception while listening to a Japanese word (2- or 3-mora in length) played repeatedly for 5 min. In the auditory memory task, a different sequence of three morae, e.g., /ka/, /hi/, and /su/, was presented to each ear simultaneously. After some period (0-4 s), participants were visually cued to recall one of the sequences, i.e., in the left or right ear. We found that delayed recall accuracy was negatively correlated with the number of VTs, particularly under 2-mora conditions. This suggests that memory persistence is important for formation and selection of perceptual objects.

Articulation posture influences pitch during singing imagery

Facial muscle activity contributes to singing and to articulation: in articulation, mouth shape can alter vowel identity; and in singing, facial movement correlates with pitch changes. Here, we examine whether mouth posture causally influences pitch during singing imagery. Based on perception-action theories and embodied cognition theories, we predict that mouth posture influences pitch judgments even when no overt utterances are produced. In two experiments (total N = 160), mouth posture was manipulated to resemble the articulation of either /i/ (as in English meet; retracted lips) or /o/ (as in French rose; protruded lips). Holding this mouth posture, participants were instructed to mentally "sing" given songs (which were all positive in valence) while listening with their inner ear and, afterwards, to assess the pitch of their mental chant. As predicted, compared to the o-posture, the i-posture led to higher pitch in mental singing. Thus, bodily states can shape experiential qualities, such as pitch, during imagery. This extends embodied music cognition and demonstrates a new link between language and music.

Irrelevant speech impairs serial recall of verbal but not spatial items in children and adults

Immediate serial recall of visually presented items is reliably impaired by task-irrelevant speech that the participants are instructed to ignore ("irrelevant speech effect," ISE). The ISE is stronger with changing speech tokens (words or syllables) when compared to repetitions of single tokens ("changing-state effect," CSE). These phenomena have been attributed to sound-induced diversions of attention away from the focal task (attention capture account), or to specific interference of obligatory, involuntary sound processing with either the integrity of phonological traces in a phonological short-term store (phonological loop account), or the efficiency of a domain-general rehearsal process employed for serial order retention (changing-state account). Aiming to further explore the role of attention, phonological coding, and serial order retention in the ISE, we analyzed the effects of steady-state and changing-state speech on serial order reconstruction of visually presented verbal and spatial items in children (n = 81) and adults (n = 80). In the verbal task, both age groups performed worse with changing-state speech (sequences of different syllables) when compared with steady-state speech (one syllable repeated) and silence. Children were more impaired than adults by both speech sounds. In the spatial task, no disruptive effect of irrelevant speech was found in either group. These results indicate that irrelevant speech evokes similarity-based interference, and thus pose difficulties for the attention-capture and the changing-state account of the ISE.

Intrinsic Laryngeal Muscle Activity During Subvocalization

PURPOSE

Subvocalization, the low-grade activity of speech articulator muscles while thinking or reading, may mediate phonological representations of verbal material. However, no literature exists that directly measures whether intrinsic laryngeal muscles (ILMs) are active during subvocalization. The possibility of ILM activation during subvocalization has implications for establishing appropriate baselines when experimental conditions involve linguistic features.

METHOD

In two separate studies, forty-five cisgender women completed one or two silentsil tasks (two in the first study, Experiments 1a and 1b, and one in the second, Experiment 2). Fine wire electromyography was used to directly measure ILM activity during an at-rest baseline and silent tasks used to determine whether subvocalization occurred (referred to hereafter as "subvocalization tasks"). Other muscles were measured via surface electromyography: submental muscle in Experiments 1a and 1b, anterior tibialis in Experiment 2, and upper trapezius in all experiments.

RESULTS

Interrupted time-series analysis was used to directly measure changes in ILM activity from baseline to the subvocalization tasks. A paired two tailed t-test was used to measure mean differences in ILM activity across conditions for each participant. Some individuals displayed statistically significant increases from baseline during subvocalization tasks, whereas others displayed decreases. Cohen's d was used to calculate the effect size for each muscle across the three subvocalization conditions. Of the 21 muscles measured across three experiments, five yielded a small mean effect size, and the effect sizes for the remaining 16 muscles were negligible. At a group level, only the right cricothyroid showed statistically significant changes (Experiment 1b).

CONCLUSION

The ILM responses during subvocalization vary in both magnitude and direction. Most but not all changes can be described as negligible. For future studies of ILM activity during conditions that involve linguistic processing, investigators should consider the idiosyncratic variation during subvocalization when determining the most appropriate baseline task.

Instructed versus spontaneous entrainment of running cadence to music tempo

Matching exercise behavior to musical beats has been shown to favorably affect repetitive endurance tasks. In this study, our aim was to explore the role of spontaneous versus instructed entrainment, focusing on self‐paced exercise of healthy, recreational runners. For three 4‐min running tasks, 33 recreational participants were either running in silence or with music; when running with music, either no instructions were given to entrain to the music, or participants were instructed to match their running cadence with the tempo of the music. The results indicated that less entrainment occurred when no instruction to match the exercise with the musical tempo was provided. In addition, similar to the condition without music, lower speeds and shorter step lengths were observed when runners were instructed to match their running behavior to the musical tempo when compared with the condition without such instruction. Our findings demonstrate the impact of instruction on running performance and stress the importance of intention to entrain running behavior to musical beats.

Mental Effort When Playing, Listening, and Imagining Music in One Pianist's Eyes and Brain

We investigated "musical effort" with an internationally renowned, classical, pianist while playing, listening, and imagining music. We used pupillometry as an objective measure of mental effort and fMRI as an exploratory method of effort with the same musical pieces. We also compared a group of non-professional pianists and non-musicians by the use of pupillometry and a small group of non-musicians with fMRI. This combined approach of psychophysiology and neuroimaging revealed the cognitive work during different musical activities. We found that pupil diameters were largest when "playing" (regardless of whether there was sound produced or not) compared to conditions with no movement (i.e., "listening" and "imagery"). We found positive correlations between pupil diameters of the professional pianist during different conditions with the same piano piece (i.e., normal playing, silenced playing, listen, imagining), which might indicate similar degrees of load on cognitive resources as well as an intimate link between the motor imagery of sound-producing body motions and gestures. We also confirmed that musical imagery had a strong commonality with music listening in both pianists and musically naïve individuals. Neuroimaging provided evidence for a relationship between noradrenergic (NE) activity and mental workload or attentional intensity within the domain of music cognition. We found effort related activity in the superior part of the locus coeruleus (LC) and, similarly to the pupil, the listening and imagery engaged less the LC-NE network than the motor condition. The pianists attended more intensively to the most difficult piece than the non-musicians since they showed larger pupils for the most difficult piece. Non-musicians were the most engaged by the music listening task, suggesting that the amount of attention allocated for the same task may follow a hierarchy of expertise demanding less attentional effort in expert or performers than in novices. In the professional pianist, we found only weak evidence for a commonality between subjective effort (as rated measure-by-measure) and the objective effort gauged with pupil diameter during listening. We suggest that psychophysiological methods like pupillometry can index mental effort in a manner that is not available to subjective awareness or introspection.

The ConDialInt Model: Condensation, Dialogality, and Intentionality Dimensions of Inner Speech Within a Hierarchical Predictive Control Framework

Inner speech has been shown to vary in form along several dimensions. Along condensation, condensed inner speech forms have been described, that are supposed to be deprived of acoustic, phonological and even syntactic qualities. Expanded forms, on the other extreme, display articulatory and auditory properties. Along dialogality, inner speech can be monologal, when we engage in internal soliloquy, or dialogal, when we recall past conversations or imagine future dialogs involving our own voice as well as that of others addressing us. Along intentionality, it can be intentional (when we deliberately rehearse material in short-term memory) or it can arise unintentionally (during mind wandering). We introduce the ConDialInt model, a neurocognitive predictive control model of inner speech that accounts for its varieties along these three dimensions. ConDialInt spells out the condensation dimension by including inhibitory control at the conceptualization, formulation or articulatory planning stage. It accounts for dialogality, by assuming internal model adaptations and by speculating on neural processes underlying perspective switching. It explains the differences between intentional and spontaneous varieties in terms of monitoring. We present an fMRI study in which we probed varieties of inner speech along dialogality and intentionality, to examine the validity of the neuroanatomical correlates posited in ConDialInt. Condensation was also informally tackled. Our data support the hypothesis that expanded inner speech recruits speech production processes down to articulatory planning, resulting in a predicted signal, the inner voice, with auditory qualities. Along dialogality, covertly using an avatar's voice resulted in the activation of right hemisphere homologs of the regions involved in internal own-voice soliloquy and in reduced cerebellar activation, consistent with internal model adaptation. Switching from first-person to third-person perspective resulted in activations in precuneus and parietal lobules. Along intentionality, compared with intentional inner speech, mind wandering with inner speech episodes was associated with greater bilateral inferior frontal activation and decreased activation in left temporal regions. This is consistent with the reported subjective evanescence and presumably reflects condensation processes. Our results provide neuroanatomical evidence compatible with predictive control and in favor of the assumptions made in the ConDialInt model.

Pitch-specific contributions of auditory imagery and auditory memory in vocal pitch imitation

Vocal imitation guides both music and language development. Despite the developmental significance of this behavior, a sizable minority of individuals are inaccurate at vocal pitch imitation. Although previous research suggested that inaccurate pitch imitation results from deficient sensorimotor associations between pitch perception and vocal motor planning, the cognitive processes involved in sensorimotor translation are not clearly defined. In the present research, we investigated the roles of basic cognitive processes in the vocal imitation of pitch, as well as the degree to which these processes rely on pitch-specific resources. In the present study, participants completed a battery of pitch and verbal tasks to measure pitch perception, pitch and verbal auditory imagery, pitch and verbal auditory short-term memory, and pitch imitation ability. Information on participants' music background was collected, as well. Pitch imagery, pitch short-term memory, pitch discrimination ability, and musical experience were unique predictors of pitch imitation ability. Furthermore, pitch imagery was a partial mediator of the relationship between pitch short-term memory and pitch imitation ability. These results indicate that vocal imitation recruits cognitive processes that rely on at least partially separate neural resources for pitch and verbal representations.

Covert singing in anticipatory auditory imagery

To date, several fMRI studies reveal activation in motor planning areas during musical auditory imagery. We addressed whether such activations may give rise to peripheral motor activity, termed subvocalization or covert singing, using surface electromyography. Sensors placed on extrinsic laryngeal muscles, facial muscles, and a control site on the bicep measured muscle activity during auditory imagery that preceded singing, as well as during the completion of a visual imagery task. Greater activation was found in laryngeal and lip muscles for auditory than for visual imagery tasks, whereas no differences across tasks were found for other sensors. Furthermore, less accurate singers exhibited greater laryngeal activity during auditory imagery than did more accurate singers. This suggests that subvocalization may be used as a strategy to facilitate auditory imagery, which appears to be degraded in inaccurate singers. Taken together, these results suggest that subvocalization may play a role in anticipatory auditory imagery, and possibly as a way of supplementing motor associations with auditory imagery.

Discriminating autism and language impairment and specific language impairment through acuity of musical imagery

Deficits in auditory short-term memory have been widely reported in children with Specific Language Impairment (SLI), and recent evidence suggests that children with Autism Spectrum Disorder and co-morbid language impairment (ALI) experience similar difficulties. Music, like language relies on auditory memory and the aim of the study was to extend work investigating the impact of auditory short-term memory impairments to musical perception in children with neurodevelopmental disorders. Groups of children with SLI and ALI were matched on chronological age (CA), receptive vocabulary, non-verbal intelligence and digit span, and compared with CA matched typically developing (TD) controls, on tests of pitch and temporal acuity within a voluntary musical imagery paradigm. The SLI participants performed at significantly lower levels than the ALI and TD groups on both conditions of the task and their musical imagery and digit span scores were positively correlated. In contrast ALI participants performed as well as TD controls on the tempo condition and better than TD controls on the pitch condition of the task. Whilst auditory short-term memory and receptive vocabulary impairments were similar across ALI and SLI groups, these were not associated with a deficit in voluntary musical imagery performance in the ALI group.

Hearing it right: Evidence of hemispheric lateralization in auditory imagery

An advantage of the right ear (REA) in auditory processing (especially for verbal content) has been firmly established in decades of behavioral, electrophysiological and neuroimaging research. The laterality of auditory imagery, however, has received little attention, despite its potential relevance for the understanding of auditory hallucinations and related phenomena. In Experiments 1-4 we find that right-handed participants required to imagine hearing a voice or a sound unilaterally show a strong population bias to localize the self-generated auditory image at their right ear, likely the result of left-hemispheric dominance in auditory processing. In Experiments 5-8 - by means of the same paradigm - it was also ascertained that the right-ear bias for hearing imagined voices depends just on auditory attention mechanisms, as biases due to other factors (i.e., lateralized movements) were controlled. These results, suggesting a central role of the left hemisphere in auditory imagery, demonstrate that brain asymmetries can drive strong lateral biases in mental imagery.

Cognitive Expertise: An ALE Meta-Analysis

Expert performance constitutes the endpoint of skill acquisition and is accompanied by widespread neuroplastic changes. To reveal common mechanisms of reorganization associated with long-term expertise in a cognitive domain (mental calculation, chess, language, memory, music without motor involvement), we used activation likelihood estimation meta-analysis and compared brain activation of experts to nonexperts. Twenty-six studies matched inclusion criteria, most of which reported an increase and not a decrease of activation foci in experts. Increased activation occurred in the left rolandic operculum (OP 4) and left primary auditory cortex and in bilateral premotor cortex in studies that used auditory stimulation. In studies with visual stimulation, experts showed enhanced activation in the right inferior parietal cortex (area PGp) and the right lingual gyrus. Experts' brain activation patterns seem to be characterized by enhanced or additional activity in domain-specific primary, association, and motor structures, confirming that learning is localized and very specialized.

Rapid Communication: Want to block earworms from conscious awareness? B(u)y gum!

Three experiments examine the role of articulatory motor planning in experiencing an involuntary musical recollection (an “earworm”). Experiment 1 shows that interfering with articulatory motor programming by chewing gum reduces both the number of voluntary and the number of involuntary—unwanted—musical thoughts. This is consistent with other findings that chewing gum interferes with voluntary processes such as recollections from verbal memory, the interpretation of ambiguous auditory images, and the scanning of familiar melodies, but is not predicted by theories of thought suppression, which assume that suppression is made more difficult by concurrent tasks or cognitive loads. Experiment 2 shows that chewing the gum affects the experience of “hearing” the music and cannot be ascribed to a general effect on thinking about a tune only in abstract terms. Experiment 3 confirms that the reduction of musical recollections by chewing gum is not the consequence of a general attentional or dual-task demand. The data support a link between articulatory motor programming and the appearance in consciousness of both voluntary and unwanted musical recollections.

Broca's area network in language function: a pooling-data connectivity study

Background and Objective: Modern neuroimaging developments have demonstrated that cognitive functions correlate with brain networks rather than specific areas. The purpose of this paper was to analyze the connectivity of Broca’s area based on language tasks.

Methods: A connectivity modeling study was performed by pooling data of Broca’s activation in language tasks. Fifty-seven papers that included 883 subjects in 84 experiments were analyzed. Analysis of Likelihood Estimates of pooled data was utilized to generate the map; thresholds at p < 0.01 were corrected for multiple comparisons and false discovery rate. Resulting images were co-registered into MNI standard space.

Results: A network consisting of 16 clusters of activation was obtained. Main clusters were located in the frontal operculum, left posterior temporal region, supplementary motor area, and the parietal lobe. Less common clusters were seen in the sub-cortical structures including the left thalamus, left putamen, secondary visual areas, and the right cerebellum.

Conclusion: Broca’s area-44-related networks involved in language processing were demonstrated utilizing a pooling-data connectivity study. Significance, interpretation, and limitations of the results are discussed.

Sound representation in higher language areas during language generation

How language is encoded by neural activity in the higher-level language areas of humans is still largely unknown. We investigated whether the electrophysiological activity of Broca's area correlates with the sound of the utterances produced. During speech perception, the electric cortical activity of the auditory areas correlates with the sound envelope of the utterances. In our experiment, we compared the electrocorticogram recorded during awake neurosurgical operations in Broca's area and in the dominant temporal lobe with the sound envelope of single words versus sentences read aloud or mentally by the patients. Our results indicate that the electrocorticogram correlates with the sound envelope of the utterances, starting before any sound is produced and even in the absence of speech, when the patient is reading mentally. No correlations were found when the electrocorticogram was recorded in the superior parietal gyrus, an area not directly involved in language generation, or in Broca's area when the participants were executing a repetitive motor task, which did not include any linguistic content, with their dominant hand. The distribution of suprathreshold correlations across frequencies of cortical activities varied whether the sound envelope derived from words or sentences. Our results suggest the activity of language areas is organized by sound when language is generated before any utterance is produced or heard.

Blocking mimicry makes true and false smiles look the same

Recent research suggests that facial mimicry underlies accurate interpretation of subtle facial expressions. In three experiments, we manipulated mimicry and tested its role in judgments of the genuineness of true and false smiles. Experiment 1 used facial EMG to show that a new mouthguard technique for blocking mimicry modifies both the amount and the time course of facial reactions. In Experiments 2 and 3, participants rated true and false smiles either while wearing mouthguards or when allowed to freely mimic the smiles with or without additional distraction, namely holding a squeeze ball or wearing a finger-cuff heart rate monitor. Results showed that blocking mimicry compromised the decoding of true and false smiles such that they were judged as equally genuine. Together the experiments highlight the role of facial mimicry in judging subtle meanings of facial expressions.

Neural correlates of auditory temporal predictions during sensorimotor synchronization

Musical ensemble performance requires temporally precise interpersonal action coordination. To play in synchrony, ensemble musicians presumably rely on anticipatory mechanisms that enable them to predict the timing of sounds produced by co-performers. Previous studies have shown that individuals differ in their ability to predict upcoming tempo changes in paced finger-tapping tasks (indexed by cross-correlations between tap timing and pacing events) and that the degree of such prediction influences the accuracy of sensorimotor synchronization (SMS) and interpersonal coordination in dyadic tapping tasks. The current functional magnetic resonance imaging study investigated the neural correlates of auditory temporal predictions during SMS in a within-subject design. Hemodynamic responses were recorded from 18 musicians while they tapped in synchrony with auditory sequences containing gradual tempo changes under conditions of varying cognitive load (achieved by a simultaneous visual n-back working-memory task comprising three levels of difficulty: observation only, 1-back, and 2-back object comparisons). Prediction ability during SMS decreased with increasing cognitive load. Results of a parametric analysis revealed that the generation of auditory temporal predictions during SMS recruits (1) a distributed network of cortico-cerebellar motor-related brain areas (left dorsal premotor and motor cortex, right lateral cerebellum, SMA proper and bilateral inferior parietal cortex) and (2) medial cortical areas (medial prefrontal cortex, posterior cingulate cortex). While the first network is presumably involved in basic sensory prediction, sensorimotor integration, motor timing, and temporal adaptation, activation in the second set of areas may be related to higher-level social-cognitive processes elicited during action coordination with auditory signals that resemble music performed by human agents.

How silent is silent reading? Intracerebral evidence for top-down activation of temporal voice areas during reading

As you might experience it while reading this sentence, silent reading often involves an imagery speech component: we can hear our own "inner voice" pronouncing words mentally. Recent functional magnetic resonance imaging studies have associated that component with increased metabolic activity in the auditory cortex, including voice-selective areas. It remains to be determined, however, whether this activation arises automatically from early bottom-up visual inputs or whether it depends on late top-down control processes modulated by task demands. To answer this question, we collaborated with four epileptic human patients recorded with intracranial electrodes in the auditory cortex for therapeutic purposes, and measured high-frequency (50-150 Hz) "gamma" activity as a proxy of population level spiking activity. Temporal voice-selective areas (TVAs) were identified with an auditory localizer task and monitored as participants viewed words flashed on screen. We compared neural responses depending on whether words were attended or ignored and found a significant increase of neural activity in response to words, strongly enhanced by attention. In one of the patients, we could record that response at 800 ms in TVAs, but also at 700 ms in the primary auditory cortex and at 300 ms in the ventral occipital temporal cortex. Furthermore, single-trial analysis revealed a considerable jitter between activation peaks in visual and auditory cortices. Altogether, our results demonstrate that the multimodal mental experience of reading is in fact a heterogeneous complex of asynchronous neural responses, and that auditory and visual modalities often process distinct temporal frames of our environment at the same time.

Mental imagery for musical changes in loudness

Musicians imagine music during mental rehearsal, when reading from a score, and while composing. An important characteristic of music is its temporality. Among the parameters that vary through time is sound intensity, perceived as patterns of loudness. Studies of mental imagery for melodies (i.e., pitch and rhythm) show interference from concurrent musical pitch and verbal tasks, but how we represent musical changes in loudness is unclear. Theories suggest that our perceptions of loudness change relate to our perceptions of force or effort, implying a motor representation. An experiment was conducted to investigate the modalities that contribute to imagery for loudness change. Musicians performed a within-subjects loudness change recall task, comprising 48 trials. First, participants heard a musical scale played with varying patterns of loudness, which they were asked to remember. There followed an empty interval of 8 s (nil distractor control), or the presentation of a series of four sine tones, or four visual letters or three conductor gestures, also to be remembered. Participants then saw an unfolding score of the notes of the scale, during which they were to imagine the corresponding scale in their mind while adjusting a slider to indicate the imagined changes in loudness. Finally, participants performed a recognition task of the tone, letter, or gesture sequence. Based on the motor hypothesis, we predicted that observing and remembering conductor gestures would impair loudness change scale recall, while observing and remembering tone or letter string stimuli would not. Results support this prediction, with loudness change recalled less accurately in the gestures condition than in the control condition. An effect of musical training suggests that auditory and motor imagery ability may be closely related to domain expertise.

Influences of multiple memory systems on auditory mental image acuity

The influence of different memory systems and associated attentional processes on the acuity of auditory images, formed for the purpose of making intonation judgments, was examined across three experiments using three different task types (cued-attention, imagery, and two-tone discrimination). In experiment 1 the influence of implicit long-term memory for musical scale structure was manipulated by varying the scale degree (leading tone versus tonic) of the probe note about which a judgment had to be made. In experiments 2 and 3 the ability of short-term absolute pitch knowledge to develop was manipulated by presenting blocks of trials in the same key or in seven different keys. The acuity of auditory images depended on all of these manipulations. Within individual listeners, thresholds in the two-tone discrimination and cued-attention conditions were closely related. In many listeners, cued-attention thresholds were similar to thresholds in the imagery condition, and depended on the amount of training individual listeners had in playing a musical instrument. The results indicate that mental images formed at a sensory/cognitive interface for the purpose of making perceptual decisions are highly malleable.

Effects of discrepancy between imagined and perceived sounds on the N2 component of the event-related potential

Two experiments were conducted to examine whether the N2 component of the event-related potential (ERP), typically elicited in a S1-S2 matching task and considered to reflect mismatch process, can still be elicited when the S1 was imagined instead of perceived and to investigate how N2 amplitude varied with the degree of S1-S2 discrepancy. Three levels of discrepancy were defined by the degree of separation between the heard (S2) and imagined (S1) sounds. It was found that the N2 was reliably elicited when the perceived S2 differed from the imagined S1, but whether N2 amplitude increased with the degree of discrepancy depended in part on the S1-S2 discriminability (as evidenced by reaction time). Specifically, the effect of increasing discrepancy was attenuated as discriminability increased from hard to easy. These results, together with the dynamic ERP topography observed within the N2 window, suggest that the N2 effect reflects two sequential but overlapping processes: automatic mismatch and controlled detection.

Parieto-frontal gamma band activity during the perceptual emergence of speech forms

The multistable perception of speech refers to the perceptual changes experienced while listening to a speech form cycled in rapid and continuous repetition, the so-called Verbal Transformation Effect. Because distinct interpretations of the same repeated stimulus alternate spontaneously, this effect provides an invaluable tool to examine how speech percepts are formed in the listener's mind. In order to track the temporal dynamics of brain activity specifically linked to perceptual changes, intracerebral EEG activity was recorded from two implanted epileptic patients while performing a verbal transformation task. To this aim, they were asked to carefully listen to a speech sequence played repeatedly and to press a button whenever they perceived a change in the repeated utterance. For both patients, 300-800 ms prior to the reported perceptual transitions, high frequency activity in the gamma band range (>40 Hz) was observed within the left inferior frontal and supramarginal gyri. An additional auditory decision task was used to rule out the possibility that the increased gamma band activity was due to the patients' motor responses. These results suggest that articulatory-based representations play a key part in the endogenously driven emergence of auditory speech percepts. The findings are interpreted in relation to theories assuming a link between perception and action in the human speech processing system.

Neural correlates of consolidation in working memory

Many of our daily activities rely on a brain system called working memory, which implements our ability to encode information for short-term maintenance, possible manipulation, and retrieval. A recent intracranial study of patients performing a paradigmatic working memory task revealed that the maintenance of information involves a distributed network of oscillations in the gamma band (>40 Hz). Using a similar task, we focused on the encoding stage and targeted a process referred to as short-term consolidation, which corresponds to the encoding of novel items in working memory. The paradigm was designed to manipulate the subjects' intention to encode: series of 10 letters were presented, among which only five had to be remembered, as indicated by visual cues preceding or following each letter. During this task we recorded the intracerebral EEG of nine epileptic patients implanted in mesiotemporal structures, perisylvian regions, and prefrontal areas and used time-frequency analysis to search for neural activities simultaneous with the encoding of the letters into working memory. We found such activities in the form of increases of gamma band activity in a set of regions associated with the phonological loop, including the Broca area and the auditory cortex, and in the prefrontal cortex, the pre- and postcentral gyri, the hippocampus, and the fusiform gyrus.

Event-related potentials during mental imagery of animal sounds

To investigate the neural correlates of imagined animal sounds, event-related potentials (ERPs) were recorded while subjects were presented with (1) animal pictures without any imagery instruction (control) or (2) animal pictures with instructions to imagine the corresponding sounds (imagery). The results revealed imagery effects starting with an enhancement of the P2, possibly indexing the top-down allocation of attention to the imagery task, and continuing into a more positive-going deflection in the time window of 350-600 ms poststimulus, probably reflecting the formation of auditory imagery. A centro-parietally distributed late positive complex (LPC) was identified in the difference waveform (imagery minus control) and might reflect two subprocesses of imagery formation: sound retrieval from stored information and representation in working memory.

Multistable syllables as enacted percepts: a source of an asymmetric bias in the verbal transformation effect

Perceptual changes are experienced during rapid and continuous repetition of a speech form, leading to an auditory illusion known as the verbal transformation effect. Although verbal transformations are considered to reflect mainly the perceptual organization and interpretation of speech, the present study was designed to test whether or not speech production constraints may participate in the emergence of verbal representations. With this goal in mind, we examined whether variations in the articulatory cohesion of repeated nonsense words--specifically, temporal relationships between articulatory events--could lead to perceptual asymmetries in verbal transformations. The first experiment displayed variations in timing relations between two consonantal gestures embedded in various nonsense syllables in a repetitive speech production task. In the second experiment, French participants repeatedly uttered these syllables while searching for verbal transformation. Syllable transformation frequencies followed the temporal clustering between consonantal gestures: The more synchronized the gestures, the more stable and attractive the syllable. In the third experiment, which involved a covert repetition mode, the pattern was maintained without external speech movements. However, when a purely perceptual condition was used in a fourth experiment, the previously observed perceptual asymmetries of verbal transformations disappeared. These experiments demonstrate the existence of an asymmetric bias in the verbal transformation effect linked to articulatory control constraints. The persistence of this effect from an overt to a covert repetition procedure provides evidence that articulatory stability constraints originating from the action system may be involved in auditory imagery. The absence of the asymmetric bias during a purely auditory procedure rules out perceptual mechanisms as a possible explanation of the observed asymmetries.

Multistable representation of speech forms: a functional MRI study of verbal transformations

We used functional magnetic resonance imaging (fMRI) to localize the brain areas involved in the imagery analogue of the verbal transformation effect, that is, the perceptual changes that occur when a speech form is cycled in rapid and continuous mental repetition. Two conditions were contrasted: a baseline condition involving the simple mental repetition of speech sequences, and a verbal transformation condition involving the mental repetition of the same items with an active search for verbal transformation. Our results reveal a predominantly left-lateralized network of cerebral regions activated by the verbal transformation task, similar to the neural network involved in verbal working memory: the left inferior frontal gyrus, the left supramarginal gyrus, the left superior temporal gyrus, the anterior part of the right cingulate cortex, and the cerebellar cortex, bilaterally. Our results strongly suggest that the imagery analogue of the verbal transformation effect, which requires percept analysis, form interpretation, and attentional maintenance of verbal material, relies on a working memory module sharing common components of speech perception and speech production systems.

The cognitive neuropsychiatry of auditory verbal hallucinations: an overview

INTRODUCTION

The cognitive neuropsychiatric approach to auditory verbal hallucinations (AVHs) attempts to explain the phenomena in cognitive or information-processing terms and ultimately their brain bases.

METHODS

A narrative review of the literature and an overview of this special issue of Cognitive Neuropsychiatry.

RESULTS

First, an operational definition of AVHs is offered. Next, clues to etiology are derived from a detailed consideration of the clinical phenomenology of "voices", their form and content. Functional and structural neuroimaging studies suggest the importance of left-side language areas in the generation/perception of AVHs.

CONCLUSIONS

Existing cognitive neuropsychiatric models provide a useful framework for the understanding of AVHs. However, data need to be applied more specifically to these models so that they may be refined.

The articulatory loop: study of the subcortical connectivity by electrostimulation

Although a cortical network involving Broca's area and the supramarginal gyrus (SMG) was widely studied using neurofunctional imaging, the functional connectivity underlying this so-called articulatory loop remains poorly documented. We describe a patient operated on for a glioma invading the left parietal operculum, using intraoperative electrical functional mapping under local anesthesia. Following the identification of cortical language sites within Broca's area and SMG, the subcortical pathways connecting these regions were detected and preserved during the resection. Postoperatively, the patient presented a slight dysarthria, then recovered. This is the first report of direct tracking of the subcortical connectivity underlying the fronto-parietal articulatory loop, allowing to better understand the pathophysiology of this network and the consequences of its damage.

Human brain mapping of auditory imagery: event-related functional MRI study

We used event-related fMRI methodology to investigate human brain activity during auditory imagery. A series of susceptibility-weighted MR images covering the whole brain were acquired to obtain blood oxygenation level-dependent (BOLD) signal changes associated with the imagery event of hearing simple monotone. Group analysis across the 12 right-handed subjects revealed activations in the medial and inferior frontal gyri, precuneus, middle frontal gyri, superior temporal gyri, and anterior cingulate gyri. Bilateral primary and secondary auditory areas in the superior temporal gyri also exhibited the event-related MR signal changes. The proposed method allowed for the analysis of brain areas responsive to the event of auditory imagery while our results suggest that auditory imagery and actual audition share common neural substrates.

Perceiving imitatible stimuli: consequences of isomorphism between input and output

For more than a century, psychologists have been intrigued by the idea that mental representations of perceived human actions are closely connected with mental representations of performing those same actions. In this article, connections between input and output representations are considered in terms of the potential for imitation. A broad range of evidence suggests that, for imitatible stimuli, input and output representations are isomorphic to one another, allowing mutual influence between perception and motoric planning that is rapid, effortless, and possibly obligatory. Thus, the cognitive consequences of imitatibility may underlie such diverse phenomena as phoneme perception; imitation in neonates; echoic memory; stimulus-response compatibility; conduction aphasia; maintenance rehearsal; and a variety of developmental and social activities such as language acquisition, social learning, empathy, and monitoring one's own behavior.

Cerebral substrates of musical imagery

Musical imagery refers to the experience of "replaying" music by imagining it inside the head. Whereas visual imagery has been extensively studied, few people have investigated imagery in the auditory domain. This article reviews a program of research that has tried to characterize auditory imagery for music using both behavioral and cognitive neuroscientific tools. I begin by describing some of my behavioral studies of the mental analogues of musical tempo, pitch, and temporal extent. I then describe four studies using three techniques that examine the correspondence of brain involvement in actually perceiving vs. imagining familiar music. These involve one lesion study with epilepsy surgery patients, two positron emission tomography (PET) studies, and one study using transcranial magnetic stimulation (TMS). The studies converge on the importance of the right temporal neocortex and other right-hemisphere structures in the processing of both perceived and imagined nonverbal music. Perceiving and imagining songs that have words also involve structures in the left hemisphere. The supplementary motor area (SMA) is activated during musical imagery; it may mediate rehearsal that involves motor programs, such as imagined humming. Future studies are suggested that would involve imagery of sounds that cannot be produced by the vocal tract to clarify the role of the SMA in auditory imagery.

The case for sensorimotor coding in working memory

The highly influential Baddeley and Hitch model of working memory (Baddeley & Hitch, 1974; see also Baddeley, 1986) posited analogical forms of representation that can be broadly characterized as sensorimotor, both for verbal and for visuospatial material. However, difficulties with the model of verbal working memory in particular have led investigators to develop alternative models that avoid appealing either to sensory coding or to motoric coding, or to both. This paper examines the evidence for sensorimotor coding in working memory, including evidence from neuropsychology and from sign language research, as well as from standard working memory paradigms, and concludes that only a sensorimotor model can accommodate the broad range of effects that characterize verbal working memory. In addition, several findings that have been considered to speak against sensorimotor involvement are reexamined and are argued to be in fact compatible with sensorimotor coding. These conclusions have broad implications, in that they support the emerging theoretical viewpoint of embodied cognition.

Is overt stuttered speech a prerequisite for the neural activations associated with chronic developmental stuttering?

Four adult right-handed chronic stutterers and four age-matched controls completed H(2)(15)O PET scans involving overt and imagined oral reading tasks. During overt stuttered speech prominent activations occurred in SMA (medial), BA 46 (right), anterior insula (bilateral), and cerebellum (bilateral) plus deactivations in right A2 (BA 21/22). These activations and deactivations also occurred when the same stutterers imagined they were stuttering. Some parietal regions were significantly activated during imagined stuttering, but not during overt stuttering. Most regional activations changed in the same direction when overt stuttering ceased (during chorus reading) and when subjects imagined that they were not stuttering (also during chorus reading). Controls displayed fewer similarities between regional activations and deactivations during actual and imagined oral reading. Thus overt stuttering appears not to be a prerequisite for the prominent regional activations and deactivations associated with stuttering.

The role of subvocalization in rehearsal and maintenance of rhythmic patterns

This experiment analyzed the influence of subvocal activity in retention of rhythmical auditory patterns. Retention of sixteen percussion sequences was studied. Each sequence (a 4-s "door-knocking" pattern) was followed by one of the following six retention conditions: silence, unattended music (blocking the inner ear, i.e., Gregorian chanting), unattended music (blocking the inner ear, i.e., rock-and-roll), articulatory suppression (blocking the inner voice), tracing circles on the table with index finger (spatial task), and tapping (motor control). After silence, unattended music (chanting), or the spatial task, participants successfully reproduced most patterns. Errors increased with unattended music (rock-and-roll), but significant disruptions only occurred with tapping and articulatory suppression. Whereas the latter case supports the role of an articulatory loop in retention, the production of successive taps or syllables in both interference conditions probably relies on a general rhythm module, which disrupted retention of the patterns.

Generative character of perception: a neural architecture for sensorimotor anticipation

The basic idea of our anticipatory approach to perception is to avoid the common separation of perception and generation of behavior and to fuse both aspects into a consistent neural process. Our approach tries to explain the phenomenon of perception, in particular, of perception at the level of sensorimotor intelligence, from a behavior-oriented point of view. Perception is assumed to be a generative process of anticipating the course of events resulting from alternative sequences of hypothetically executed actions. By means of this sensorimotor anticipation, it is possible to characterize a visual scenery immediately in categories of behavior, i.e. by a set of actions which describe possible methods of interaction with the objects in the environment. Thus, the competence to perceive a complex situation can be understood as the capability to anticipate the course of events caused by different action sequences. Starting from an abstract description of anticipatory perception and the essential biological evidence for internal simulation, we present two biologically motivated computational models that are able to anticipate and evaluate hypothetically sensorimotor sequences. Both models consider functional aspects of those cortical and subcortical systems that are assumed to be involved in the process of sensory prediction and sensorimotor control. Our first approach, the Model for Anticipation based on Sensory IMagination (MASIM), realizes a sequential search in sensorimotor space using a simple model of lateral cerebellum as sensory predictor. We demonstrate the efficiency of this model approach in the light of visually guided local navigation behaviors of a mobile system. The second approach, the Model for Anticipation based on Cortical Representations (MACOR), is actually still at a conceptual level of realization. We postulate that this model allows a completely parallel search at the neocortical level using assemblies of spiking neurons for grouping, separation, and selection of sensorimotor sequences. Both models are intended as general schemes for anticipation based perception at the level of sensorimotor intelligence.

Effect of training on the ability of dual-task coordination

Within the framework of the working memory model proposed by A. Baddeley and G. Hitch, a dual-task paradigm has been suggested to evaluate the capacity to perform simultaneously two concurrent tasks. This capacity is assumed to reflect the functioning of the central executive component, which appears to be impaired in patients with dysexecutive syndrome. The present study extends the investigation of an index ("mu"), which is supposed to indicate the capacity of coordination of concurrent auditory digit span and tracking tasks, by testing the influence of training on the performance in the dual task. The presentation of the same digit sequence lists or always-different lists did not differently affect the performance. The span length affected the mu values. The improved performance in the tasks under the dual condition closely resembled the improvement in the single-task performance. So, although training improved performance in the single and dual conditions, especially for the tracking component, the mu values remained stable throughout the sessions when the single tasks were performed first. Conversely, training improved the capacity of dual-task coordination throughout the sessions when dual task was performed first, addressing the issue of the contribution of the within-session practice to the mu index.

A Cortical-type Modular Neural Network for Hypothetical Reasoning

We propose a multilayer neural network architecture that can implement the kind of hypothetical reasoning that the cortex seems to perform in making sense of the sensory input. The elementary processing nodes of each homogeneous sheet are not single formal neurons, but complex modules abducted from the functional organization of neocortical columns. As an example, we simulate face recognition in this neocortical architecture. A holistic but coarse initial hypothesis is generated by express forward input description and subsequently refined under the constraints of this hypothesis. Separation of forward input description and feedback generated hypothesis, while using the difference in both descriptions at each of the modular units to control the refinement, enables robust recognition and has the potential for autonomous learning. Copyright 1997 Elsevier Science Ltd.