Interpreting chicken-scratch: lexical access for handwritten words

Breaking the boundaries: the power of ligatures in visual-word recognition

Introduction

Current neurobiological-inspired models of visual-word recognition propose that letter detectors in the word recognition system can tolerate some variations in the visual form of the letters. However, it is unclear whether this tolerance extends to novel ligatures, which combine two letters into a single glyph.

Methods

To investigate this, the present study utilized a masked priming experiment with a lexical decision task to examine whether primes containing novel ligatures are effective in activating their corresponding base word, relative to omitted-letter primes, in the initial stages of word processing. For each target word (e.g., VIRTUAL), were created an identity prime (virtual), a prime containing a novel ligature of two of the letters (e.g., virtual; "ir" in a single glyph), and an omitted-letter prime where one letter was removed (e.g., vrtual [omitted-vowel] in Experiment 1; vitual [omitted-consonant] in Experiment 2).

Results

Results showed that the presence of a novel ligature in the prime resulted in faster lexical decision times compared to a prime with an omitted vowel (Experiment 1), but not with an omitted consonant (Experiment 2). Furthermore, the performance with the primes containing the novel ligature was not different from that of the identity primes.

Discussion

These results suggest that the word recognition system can quickly enable separate letter detectors for novel ligatures. These findings have important implications for our understanding of the front-end of visual-word recognition.

Rëâdīńg wõrdš wîth ōrńåmêńtš: is there a cost?

Introduction

Recent research has reported that adding non-existent diacritical marks to a word produces a minimal reading cost compared to the intact word. Here we examined whether this minimal reading cost is due to: (1) the resilience of letter detectors to the perceptual noise (i.e., the cost should be small and comparable for words and nonwords) or (2) top-down lexical processes that normalize the percept for words (i.e., the cost would be larger for nonwords).

Methods

We designed a letter detection experiment in which a target stimulus (either a word or a nonword) was presented intact or with extra non-existent diacritics [e.g., amigo (friend) vs. ãmîgô; agimo vs. ãgîmô]. Participants had to decide which of two letters was in the stimulus (e.g., A vs. U).

Results

Although the task involved lexical processing, with responses being faster and more accurate for words compared to nonwords, we found only a minimal advantage in error rates for intact stimuli versus those with non-existent diacritics. This advantage was similar for both words and nonwords.

Discussion

The letter detectors in the word recognition system appear to be resilient to non-existent diacritics without the need for feedback from higher levels of processing.

The inversion effect in word recognition: the effect of language familiarity and handwriting

Humans have expertise with visual words and faces. One marker of this expertise is the inversion effect. This is attributed to experience with those objects being biased towards a canonical orientation, rather than some inherent property of object structure or perceptual anisotropy. To confirm the role of experience, we measured inversion effects in word matching for familiar and unfamiliar languages. Second, we examined whether there may be more demands on reading expertise with handwritten stimuli rather than computer font, given the greater variability and irregularities in the former, with the prediction of larger inversion effects for handwriting. We recruited two cohorts of subjects, one fluent in Farsi and the other in Punjabi, neither of whom were able to read the other's language. Subjects performed a match-to-sample task with words in either computer fonts or handwritings. Subjects were more accurate and faster with their familiar language, even when it was inverted. Inversion effects were present for the familiar but not the unfamiliar language. The inversion effect in accuracy for handwriting was larger than that for computer fonts in the familiar language. We conclude that the word inversion effect is generated solely by orientation-biased experience, and that demands on this expertise are greater with handwriting than computer font.

Whole-object effects in visual word processing: Parallels with and differences from face recognition

Visual words and faces differ in their structural properties, but both are objects of high expertise. Holistic processing is said to characterize expert face recognition, but the extent to which whole-word processes contribute to word recognition is unclear, particularly as word recognition is thought to proceed by a component-based process. We review the evidence for experimental effects in word recognition that parallel those used to support holistic face processing, namely inversion effects, the part-whole task, and composite effects, as well as the status of whole-word processing in pure alexia and developmental dyslexia, contrasts between familiar and unfamiliar languages, and the differences between handwriting and typeset font. The observations support some parallels in whole-object influences between face and visual word recognition, but do not necessarily imply similar expert mechanisms. It remains to be determined whether and how the relative balance between part-based and whole-object processing differs for visual words and faces.

The time course of processing handwritten words: An ERP investigation

Behavioral studies have shown that the legibility of handwritten script hinders visual word recognition. Furthermore, when compared with printed words, lexical effects (e.g., word-frequency effect) are magnified for less intelligible (difficult) handwriting (Barnhart and Goldinger, 2010; Perea et al., 2016). This boost has been interpreted in terms of greater influence of top-down mechanisms during visual word recognition. In the present experiment, we registered the participants' ERPs to uncover top-down processing effects on early perceptual encoding. Participants' behavioral and EEG responses were recorded to high- and low-frequency words that varied in script's legibility (printed, easy handwritten, difficult handwritten) in a lexical decision experiment. Behavioral results replicated previous findings: word-frequency effects were larger in difficult handwriting than in easy handwritten or printed conditions. Critically, the ERP data showed an early effect of word-frequency in the N170 that was restricted to the difficult-to-read handwritten condition. These results are interpreted in terms of increased attentional deployment when the bottom-up signal is weak (difficult handwritten stimuli). This attentional boost would enhance top-down effects (e.g., lexical effects) in the early stages of visual word processing.

GazeR: A Package for Processing Gaze Position and Pupil Size Data

Eye-tracking is widely used throughout the scientific community, from vision science and psycholinguistics to marketing and human-computer interaction. Surprisingly, there is little consistency and transparency in preprocessing steps, making replicability and reproducibility difficult. To increase replicability, reproducibility, and transparency, a package in R (a free and widely used statistical programming environment) called gazeR was created to read and preprocess two types of data: gaze position and pupil size. For gaze position data, gazeR has functions for reading in raw eye-tracking data, formatting it for analysis, converting from gaze coordinates to areas of interest, and binning and aggregating data. For data from pupillometry studies, the gazeR package has functions for reading in and merging multiple raw pupil data files, removing observations with too much missing data, eliminating artifacts, blink identification and interpolation, subtractive baseline correction, and binning and aggregating data. The package is open-source and freely available for download and installation: https://github.com/dmirman/gazer . We provide step-by-step analyses of data from two tasks exemplifying the package's capabilities.

Would disfluency by any other name still be disfluent? Examining the disfluency effect with cursive handwriting

When exposed to words presented under perceptually disfluent conditions (e.g., words written in Haettenschweiler font), participants have difficulty initially recognizing the words. Those same words, though, may be better remembered later than words presented in standard type font. This counterintuitive finding is referred to as the disfluency effect. Evidence for this disfluency effect, however, has been mixed, suggesting possible moderating factors. Using a recognition memory task, level of disfluency was examined as a moderating factor across three experiments using a novel cursive manipulation that varied on degree of legibility (easy-to-read cursive vs. hard-to-read cursive). In addition, list type and retention interval between study and test were manipulated. Across all three experiments, cursive words engendered better memory than type-print words. This memory effect persisted across varied list designs (blocked vs. mixed) and a longer (24-hour) retention interval. A small-scale meta-analysis across the three experiments suggested that the cursive disfluency effect is moderated by level of disfluency: easy-to-read cursive words tended to be better remembered than hard-to-read cursive words. Taken together, these results challenge extant accounts of the disfluency effect. The theoretical and practical implications of these findings are discussed.

Is there a cost at encoding words with joined letters during visual word recognition?

For simplicity, models of visual-word recognition have focused on printed words composed of separated letters, thus overlooking the processing of cursive words. Manso de Zuniga, Humphreys, and Evett (1991) claimed that there is an early “cursive normalization” encoding stage when processing written words with joined letters. To test this claim, we conducted a lexical decision experiment in which words were presented either with separated or joined letters. To examine if the cost of letter segmentation occurs early in processing, we also manipulated a factor (i.e., word-frequency) that is posited to affect subsequent lexical processing. Results showed faster response times for the words composed of separated letters than for the words composed of joined letters. This effect occurred similarly for low- and high-frequency words. Thus, the present data offer some empirical support to Manso de Zuniga et al.’s (1991) idea of an early “cursive normalization” stage when processing joined-letters words. This pattern of data can be used to constrain the mapping of the visual input into letter and word units in future versions of models of visual word recognition.

Eye movements when reading sentences with handwritten words

The examination of how we read handwritten words (i.e., the original form of writing) has typically been disregarded in the literature on reading. Previous research using word recognition tasks has shown that lexical effects (e.g., the word-frequency effect) are magnified when reading difficult handwritten words. To examine this issue in a more ecological scenario, we registered the participants' eye movements when reading handwritten sentences that varied in the degree of legibility (i.e., sentences composed of words in easy vs. difficult handwritten style). For comparison purposes, we included a condition with printed sentences. Results showed a larger reading cost for sentences with difficult handwritten words than for sentences with easy handwritten words, which in turn showed a reading cost relative to the sentences with printed words. Critically, the effect of word frequency was greater for difficult handwritten words than for easy handwritten words or printed words in the total times on a target word, but not on first-fixation durations or gaze durations. We examine the implications of these findings for models of eye movement control in reading.

Do handwritten words magnify lexical effects in visual word recognition?

An examination of how the word recognition system is able to process handwritten words is fundamental to formulate a comprehensive model of visual word recognition. Previous research has revealed that the magnitude of lexical effects (e.g., the word-frequency effect) is greater with handwritten words than with printed words. In the present lexical decision experiments, we examined whether the quality of handwritten words moderates the recruitment of top-down feedback, as reflected in word-frequency effects. Results showed a reading cost for difficult-to-read and easy-to-read handwritten words relative to printed words. But the critical finding was that difficult-to-read handwritten words, but not easy-to-read handwritten words, showed a greater word-frequency effect than printed words. Therefore, the inherent physical variability of handwritten words does not necessarily boost the magnitude of lexical effects.

Orthographic and phonological neighborhood effects in handwritten word perception

In printed-word perception, the orthographic neighborhood effect (i.e., faster recognition of words with more neighbors) has considerable theoretical importance, because it implicates great interactivity in lexical access. Mulatti, Reynolds, and Besner Journal of Experimental Psychology: Human Perception and Performance, 32, 799-810 (2006) questioned the validity of orthographic neighborhood effects, suggesting that they reflect a confound with phonological neighborhood density. They reported that, when phonological density is controlled, orthographic neighborhood effects vanish. Conversely, phonological neighborhood effects were still evident even when controlling for orthographic neighborhood density. The present study was a replication and extension of Mulatti et al. (2006), with words presented in four different formats (computer-generated print and cursive, and handwritten print and cursive). The results from Mulatti et al. (2006) were replicated with computer-generated stimuli, but were reversed with natural stimuli. These results suggest that, when ambiguity is introduced at the level of individual letters, top-down influences from lexical neighbors are increased.

A further examination of the lexical-processing stages hypothesized by the E-Z Reader model

Participants' eye movements were monitored while they read sentences in which high- and low-frequency target words were presented normally (i.e., the normal condition) or with either reduced stimulus quality (i.e., the faint condition) or alternating lower- and uppercase letters (i.e., the case-alternated condition). Both the stimulus quality and case alternation manipulations interacted with word frequency for the gaze duration measure, such that the magnitude of word frequency effects was increased relative to the normal condition. However, stimulus quality (but not case alternation) interacted with word frequency for the early fixation time measures (i.e., first fixation, single fixation), whereas case alternation (but not stimulus quality) interacted with word frequency for the later fixation time measures (i.e., total time, go-past time). We interpret this pattern of results as evidence that stimulus quality influences an earlier stage of lexical processing than does case alternation, and we discuss the implications of our results for models of eye movement control during reading.

Rotation reveals the importance of configural cues in handwritten word perception

A dramatic perceptual asymmetry occurs when handwritten words are rotated 90 ° in either direction. Those rotated in a direction consistent with their natural tilt (typically clockwise) become much more difficult to recognize, relative to those rotated in the opposite direction. In Experiment 1, we compared computer-printed and handwritten words, all equated for degrees of leftward and rightward tilt, and verified the phenomenon: The effect of rotation was far larger for cursive words, especially when they were rotated in a tilt-consistent direction. In Experiment 2, we replicated this pattern with all items being presented in visual noise. In both experiments, word frequency effects were larger for computer-printed words and did not interact with rotation. The results suggest that handwritten word perception requires greater configural processing, relative to computer print, because handwritten letters are variable and ambiguous. When words are rotated, configural processing suffers, particularly when rotation exaggerates the natural tilt. Our account is similar to theories of the "Thatcher illusion," wherein face inversion disrupts holistic processing. Together, the findings suggest that configural, word-level processing automatically increases when people read handwriting, as letter-level processing becomes less reliable.

Word knowledge influences character perception

In two experiments, we examined whether context information can affect the activity of the nodes at the character level. Chinese readers viewed two Chinese characters; one was intact, but the other (the target) was embedded in a rectangle of visual noise and increased in visibility over time. The two characters constituted a word in one condition but did not in the other condition. The task was to press a button to indicate whether the character in the noise was at the top or bottom of the rectangle. (They did not have to identify the character.) Response times were faster in the word condition than in the nonword condition. Because the "wordness" of the stimulus was logically irrelevant to judging the location of the target character, the data indicate that processing at the word level can feed back to fairly low-level judgments, such as where a character is.

Orthographic consistency and word-frequency effects in auditory word recognition: new evidence from lexical decision and rime detection

Many studies have repeatedly shown an orthographic consistency effect in the auditory lexical decision task. Words with phonological rimes that could be spelled in multiple ways (i.e., inconsistent words) typically produce longer auditory lexical decision latencies and more errors than do words with rimes that could be spelled in only one way (i.e., consistent words). These results have been extended to different languages and tasks, suggesting that the effect is quite general and robust. Despite this growing body of evidence, some psycholinguists believe that orthographic effects on spoken language are exclusively strategic, post-lexical, or restricted to peculiar (low-frequency) words. In the present study, we manipulated consistency and word-frequency orthogonally in order to explore whether the orthographic consistency effect extends to high-frequency words. Two different tasks were used: lexical decision and rime detection. Both tasks produced reliable consistency effects for both low- and high-frequency words. Furthermore, in Experiment 1 (lexical decision), an interaction revealed a stronger consistency effect for low-frequency words than for high-frequency words, as initially predicted by Ziegler and Ferrand (1998), whereas no interaction was found in Experiment 2 (rime detection). Our results extend previous findings by showing that the orthographic consistency effect is obtained not only for low-frequency words but also for high-frequency words. Furthermore, these effects were also obtained in a rime detection task, which does not require the explicit processing of orthographic structure. Globally, our results suggest that literacy changes the way people process spoken words, even for frequent words.