Rapid extraction of lexical tone phonology in Chinese characters: a visual mismatch negativity study

Seeking the neural representation of statistical properties in print during implicit processing of visual words

Statistical learning (SL) plays a key role in literacy acquisition. Studies have increasingly revealed the influence of distributional statistical properties of words on visual word processing, including the effects of word frequency (lexical level) and mappings between orthography, phonology, and semantics (sub-lexical level). However, there has been scant evidence to directly confirm that the statistical properties contained in print can be directly characterized by neural activities. Using time-resolved representational similarity analysis (RSA), the present study examined neural representations of different types of statistical properties in visual word processing. From the perspective of predictive coding, an equal probability sequence with low built-in prediction precision and three oddball sequences with high built-in prediction precision were designed with consistent and three types of inconsistent (orthographically inconsistent, orthography-to-phonology inconsistent, and orthography-to-semantics inconsistent) Chinese characters as visual stimuli. In the three oddball sequences, consistent characters were set as the standard stimuli (probability of occurrence p = 0.75) and three types of inconsistent characters were set as deviant stimuli (p = 0.25), respectively. In the equal probability sequence, the same consistent and inconsistent characters were presented randomly with identical occurrence probability (p = 0.25). Significant neural representation activities of word frequency were observed in the equal probability sequence. By contrast, neural representations of sub-lexical statistics only emerged in oddball sequences where short-term predictions were shaped. These findings reveal that the statistical properties learned from long-term print environment continues to play a role in current word processing mechanisms and these mechanisms can be modulated by short-term predictions.

Rapid implicit extraction of abstract orthographic patterns of Chinese characters during reading

Orthographic processing is crucial in reading. For the Chinese language, sub-lexical processing has already taken place at radical level. Previous literature reported early position-specific radical representations and later position-general radical representations, implying a possible separating process of abstract position information irrespective of radicals per se from radical representations during orthographic processing. However, it remains largely unclear whether the abstract pattern of spatial arrangement of radicals can be rapidly extracted, and if so, whether this extraction takes place at the visual cortex, the very first processing center. As the visual cortex is documented to actively participate in orthographic processing, it may also play a role in the possible extraction of abstract orthographic patterns of Chinese characters. Hence, we hypothesize that abstract orthographic patterns of Chinese characters are covertly extracted at the visual cortex during reading. In this study, we investigated whether the visual cortex could rapidly extract abstract structural patterns of Chinese characters, using the event-related potential (ERP) technique. We adopted an active oddball paradigm with two types of deviant stimuli different only in one feature, structural or tonal, from standard stimuli; in each of the two sessions, subjects focused conscious attention on one feature and neglected the other. We observed that the ERPs recorded at occipital electrodes responded differentially to standard and structural deviant stimuli in both sessions, especially within the time range of the occipital P200 component. Then, we extracted three source waves arising from different levels of the visual cortex. Early response differences (from 88 to 456 ms after stimulus onset) were observed between the source waves, probably arising from left primary/secondary and bilateral associative visual cortices, in response to standard and deviant stimuli that violated abstract structural patterns, whether subjects focused their attention on the character structure or not. This suggests rapid extraction of abstract structural patterns of Chinese characters in the visual cortex, no matter the abstract structural pattern was explicit or implicit to subjects. Note that the source waves arising from right primary/secondary visual cortices in response to standard and structural deviant stimuli did not differ at all, indicating that this extraction of the abstract structural pattern of Chinese characters was left-lateralized. Besides, no difference was observed between source waves originating from any level of the visual cortex to standard and deviant stimuli that violated abstract tonal patterns, until 768 ms when a late effect related to conscious detection of targets occurred at higher levels of the visual cortex. Note that at late stages (later than 698 ms after stimulus onset), responses arising from bilateral associative visual cortices to standard and target stimuli differed for both sessions, no matter the structural or tonal feature was attended to. Our findings support the primitive intelligence of visual cortex to rapidly extract abstract orthographic patterns of Chinese characters that might be engaged in further lexical processing. Our findings also suggest that this rapid extraction can take place implicitly during reading.

Prediction errors in surface segmentation are reflected in the visual mismatch negativity, independently of task and surface features

The visual system quickly registers perceptual regularities in the environment and responds to violations in these patterns. Errors of perceptual prediction are associated with electrocortical modulation, including the visual mismatch negativity (vMMN) and P2 event-related potential. One relatively unexplored question is whether these prediction error signals can encode higher-level properties such as surface segmentation, or whether they are limited to lower-level perceptual features. Using a roving standard paradigm, a triangle surface appeared either behind (featuring amodal contours) or in front of (featuring real contours) a second surface with hole-like windows. A surface layout appeared for two to five repetitions before switching to the other "deviant" layout; lighting and orientation of stimuli varied across presentations while remaining isoluminant. Observers responded when they detected a rare "pinched" triangle, which occasionally appeared. Cortical activity-reflected in mismatch responses affecting the P2-N2 and P300 amplitudes-was sensitive to a change in stimulus layout, when surfaces shifted position in depth, following several repetitions. Specifically, layout deviants led to a more negative P2-N2 complex at posterior electrodes, and greater P300 positivity at central sites. Independently of these signals of a deviant surface layout, further modulations of the P2 encoded differences between layouts and detection of the rare target stimulus. Comparison of the effect of preceding layout repetitions on this prediction error signal suggests that it is all or none and not graded with respect to the number of previous repetitions. We show that within the visual domain, unnoticed and task-irrelevant changes in visual surface segmentation leads to observable electrophysiological signals of prediction error that are dissociable from stimulus-specific encoding and lower-level perceptual processing.

A neuromarker of clinical outcome in attention bias modification therapy for social anxiety disorder

BACKGROUND

Attention bias modification (ABM) therapy aims to modify threat-related attention patterns via computerized tasks. Despite showing medium clinical effect sizes for anxiety disorders, underlying neural-cognitive mechanisms of change remain unclear. We used visual mismatch negativity (vMMN), an event-related potential sensitive to violations of learned statistical contingencies, to assess therapy-related contingency extraction processes in healthy participants and in patients with social anxiety disorder (SAD). We then assessed whether vMMN amplitude predicts ABM treatment outcome.

METHODS

A modified version of the dot-probe task was used to elicit vMMN, in which 80% of trials were standard and 20% were deviant. In study 1, 30 healthy adults were randomly assigned to one of two ABM conditions: one in which threat-congruent targets were deviant trials and threat-incongruent targets were standard trials, and another in which the contingency was reversed. Electroencephalography (EEG) was continuously measured and vMMN analyzed. In study 2, 38 patients with SAD underwent six sessions of ABM therapy. We tested whether rule extraction in the ABM task, indicated by vMMN amplitude, predicts treatment outcome.

RESULTS

vMMN clearly emerged over prespecified scalp locations indicating contingency extraction during ABM (study 1). vMMN amplitude predicted clinical improvement after ABM therapy, uniquely accounting for 7% and 14.4% of the variance in clinician-rated and self-reported posttreatment SAD symptoms, respectively.

CONCLUSIONS

vMMN emerges as a neural marker for contingency learning in ABM, suggesting a significant role for contingency extraction processes in the clinical efficacy of this therapy.

Electrophysiological dynamics of Chinese phonology during visual word recognition in Chinese-English bilinguals

Silent word reading leads to the activation of orthographic (spelling), semantic (meaning), as well as phonological (sound) information. For bilinguals, native language information can also be activated automatically when they read words in their second language. For example, when Chinese-English bilinguals read words in their second language (English), the phonology of the Chinese translations is automatically activated. Chinese phonology, however, consists of consonants and vowels (segmental) and tonal information. To what extent these two aspects of Chinese phonology are activated is yet unclear. Here, we used behavioural measures, event-related potentials and oscillatory EEG to investigate Chinese segmental and tonal activation during word recognition. Evidence of Chinese segmental activation was found when bilinguals read English words (faster responses, reduced N400, gamma-band power reduction) and when they read Chinese words (increased LPC, gamma-band power reduction). In contrast, evidence for Chinese tonal activation was only found when bilinguals read Chinese words (gamma-band power increase). Together, our converging behavioural and electrophysiological evidence indicates that Chinese segmental information is activated during English word reading, whereas both segmental and tonal information are activated during Chinese word reading. Importantly, gamma-band oscillations are modulated differently by tonal and segmental activation, suggesting independent processing of Chinese tones and segments.

Early lexical processing of Chinese words indexed by Visual Mismatch Negativity effects

Although Mismatch Negativity (MMN) effects indicating early, automatic lexical processing have been reported in the auditory language modality, so far these have not been reliably obtained in MMN studies of visual word recognition. The present study explores this discrepancy by investigating whether visual MMN (vMMN) effects can be obtained in written Chinese single-character word recognition. While participants were engaged in a non-linguistic distraction task, we measured Event-Related Potentials (ERPs) time-locked to perifoveally-presented real and pseudo- characters matched in overall visual-orthographic attributes. VMMN was defined as significant difference between the ERPs to characters presented as deviants or as standards in a context of non-characters. For the native Chinese readers, af ter sub-lexical structural detection from 120-160 ms, only real characters elicited vMMN at the interval of 170-210 ms, suggesting that lexical information in Chinese words is processed early and automatically. In a later window of 340-380 ms, both real and pseudo- characters yielded vMMNs. In a control group of non-Chinese participants, no evidence of vMMN was found for either real or pseudo-characters. Taken together, these results suggest that long-term memory representations for real characters may enable their early processing even in unattended conditions.

Visual mismatch negativity: a predictive coding view

An increasing number of studies investigate the visual mismatch negativity (vMMN) or use the vMMN as a tool to probe various aspects of human cognition. This paper reviews the theoretical underpinnings of vMMN in the light of methodological considerations and provides recommendations for measuring and interpreting the vMMN. The following key issues are discussed from the experimentalist's point of view in a predictive coding framework: (1) experimental protocols and procedures to control "refractoriness" effects; (2) methods to control attention; (3) vMMN and veridical perception.

Processing of acoustic and phonological information of lexical tones in Mandarin Chinese revealed by mismatch negativity

The accurate perception of lexical tones in tonal languages involves the processing of both acoustic information and phonological information carried by the tonal signal. In this study we evaluated the relative role of the two types of information in native Chinese speaker’s processing of tones at a preattentive stage with event-related potentials (ERPs), particularly the mismatch negativity (MNN). Specifically, we distinguished the acoustic from the phonological information by manipulating phonological category and acoustic interval of the stimulus materials. We found a significant main effect of phonological category for the peak latency of MMN, but a main effect of both phonological category and acoustic interval for the mean amplitude of MMN. The results indicated that the two types of information, acoustic and phonological, play different roles in the processing of Chinese lexical tones: acoustic information only impacts the extent of tonal processing, while phonological information affects both the extent and the time course of tonal processing. Implications of these findings are discussed in light of neurocognitive processes of phonological processing.

Spatio-temporal dynamics of automatic processing of phonological information in visual words

Sensory-specific cortices appear to be sensitive to information from another modality. Here we investigate whether the human brain automatically extracts the phonological information in visual words in early visual processing. We continuously presented native Chinese speakers peripherally with Chinese homophone characters in an oddball paradigm, while they performed a visual detection task presented in the centre of the visual field. We found the lexical tone phonology embedded in the characters is processed automatically by the brain of native speakers, as revealed by whole-head electrical recordings of the mismatch negativity (MMN). Source solution further revealed the MMN involved the neural activations from the visual cortex to the auditory cortex (130–460 ms). The spatial-temporal dynamics indicate a visual-auditory interaction in the early, automatic processing of phonological information in visual words.

Automatic processing of unattended lexical information in visual oddball presentation: neurophysiological evidence

Previous electrophysiological studies of automatic language processing revealed early (100–200 ms) reflections of access to lexical characteristics of speech signal using the so-called mismatch negativity (MMN), a negative ERP deflection elicited by infrequent irregularities in unattended repetitive auditory stimulation. In those studies, lexical processing of spoken stimuli became manifest as an enhanced ERP in response to unattended real words, as opposed to phonologically matched but meaningless pseudoword stimuli. This lexical ERP enhancement was explained by automatic activation of word memory traces realized as distributed strongly intra-connected neuronal circuits, whose robustness guarantees memory trace activation even in the absence of attention on spoken input. Such an account would predict the automatic activation of these memory traces upon any presentation of linguistic information, irrespective of the presentation modality. As previous lexical MMN studies exclusively used auditory stimulation, we here adapted the lexical MMN paradigm to investigate early automatic lexical effects in the visual modality. In a visual oddball sequence, matched short word and pseudoword stimuli were presented tachistoscopically in perifoveal area outside the visual focus of attention, as the subjects' attention was concentrated on a concurrent non-linguistic visual dual task in the center of the screen. Using EEG, we found a visual analogue of the lexical ERP enhancement effect, with unattended written words producing larger brain response amplitudes than matched pseudowords, starting at ~100 ms. Furthermore, we also found significant visual MMN, reported here for the first time for unattended perifoveal lexical stimuli. The data suggest early automatic lexical processing of visually presented language which commences rapidly and can take place outside the focus of attention.