Object-based attention in Chinese readers of Chinese words: Beyond Gestalt principles

Unraveling the mechanism of semantic object-based attention: The role of top-down search strategies

There is an ongoing debate between the sensory enhancement account and the attentional prioritization account concerning the mechanism underlying object-based attention. This debate remains unresolved because of the predominant use of geometric objects in previous studies, which made it difficult to experimentally dissociate the two accounts due to the prominent boundaries of these objects. The current study investigated the mechanism underlying semantic object-based attention by utilizing Chinese two-character words with different word frequencies and the event-related potentials (ERPs) technique to elucidate the ongoing debate. The behavioral results showed that the semantic object effect was observed only in the high-frequency condition. The ERP results revealed the following: (1) the N1 component was larger for the high-frequency condition compared with the low-frequency condition. However, there was no significant difference in amplitude between the N1 component elicited by invalid same object locations and invalid different object locations, irrespective of whether it occurred in high- or low-frequency conditions. (2) The P3 component elicited by invalid same object locations was larger than that elicited by invalid different object locations only in the high-frequency condition. (3) The N400 amplitude elicited by invalid same object locations was smaller than that elicited by invalid different object locations only in the high-frequency condition. These results suggest that in the absence of obvious object boundaries, the production of object-based attention is primarily driven by search strategies. Attentional prioritization, rather than sensory enhancement, is the dominant mechanism underlying object-based attention.

Object-based inhibition of return in three-dimensional space: From simple drawings to real objects

Cued to an object in space, inhibition of the attended location can spread to the entire object. Although object-based inhibition of return (IOR) studies in a two-dimensional plane have been documented, the IOR has not been explored when objects cross depth in three-dimensional (3D) space. In the present study, we used a virtual reality technique to adapt the double-rectangle paradigm to a 3D space, and manipulated the cue validity and target location to examine the difference in object-based IOR between far and near spaces under different object representations. The study showed that the object-based IOR of simple drawings existed only in near space, whereas object-based IOR of real objects existed only in far space at first, and as the object similarity decreases, it appeared in both far and near spaces.

The Architecture of Object-Based Attention

The allocation of attention to objects raises several intriguing questions: What are objects, how does attention access them, what anatomical regions are involved? Here, we review recent progress in the field to determine the mechanisms underlying object-based attention. First, findings from unconscious priming and cueing suggest that the preattentive targets of object-based attention can be fully developed object representations that have reached the level of identity. Next, the control of object-based attention appears to come from ventral visual areas specialized in object analysis that project downward to early visual areas. How feedback from object areas can accurately target the object's specific locations and features is unknown but recent work in autoencoding has made this plausible. Finally, we suggest that the three classic modes of attention may not be as independent as is commonly considered, and instead could all rely on object-based attention. Specifically, studies show that attention can be allocated to the separated members of a group-without affecting the space between them-matching the defining property of feature-based attention. At the same time, object-based attention directed to a single small item has the properties of space-based attention. We outline the architecture of object-based attention, the novel predictions it brings, and discuss how it works in parallel with other attention pathways.

Faces do not guide attention in an object-based facilitation manner

Numerous studies on face processing have revealed their special ability to affect attention, but relatively little research has been done on how faces guide spatial attention allocation. To enrich this field, this study resorted to the object-based attention (OBA) effect in a modified double-rectangle paradigm where the rectangles were replaced with human faces and mosaic patterns (non-face objects). Experiment 1 replicated the typical OBA effect in the non-face objects, but this effect was absent in Asian and Caucasian faces. Experiment 2 removed the eye region from Asian faces, but still found no object-based facilitation in the faces without eyes. In Experiment 3, the OBA effect was also observed for faces when the faces disappear a short period before the responses. Overall, these results revealed that when two faces are presented together, they do not exert object-based facilitation regardless of their facial features such as race and the presence of eyes. We argue that the lack of a typical OBA effect is due to the filtering cost induced by the entire face content. This cost slows down the response when attention shifts within a face and results in the absence of object-based facilitation.

Early musical training benefits to non-musical cognitive ability associated with the Gestalt principles

Musical training has been evidenced to facilitate music perception, which refers to the consistencies, boundaries, and segmentations in pieces of music that are associated with the Gestalt principles. The current study aims to test whether musical training is beneficial to non-musical cognitive ability with Gestalt principles. Three groups of Chinese participants (with early, late, and no musical training) were compared in terms of their performances on the Motor-Free Visual Perception Test (MVPT). The results show that the participants with early musical training had significantly better performance in the Gestalt-like Visual Closure subtest than those with late and no musical training, but no significances in other Gestalt-unlike subtests was identified (Visual Memory, Visual Discrimination, Spatial Relationship, Figure Ground in MVPT). This study suggests the benefit of early musical training on non-musical cognitive ability with Gestalt principles.

The sequence effect: Character position processing in Chinese words

Numerous studies indicate that letter position processing is important for word recognition; also, the position processing of external letters (especially the initial letter) is better than that of inner letters in the Roman script. Similarly, the position processing of characters is critical in Chinese word recognition. However, the position processing pattern of characters within Chinese words is still understudied. Therefore, using a single-presentation lexical decision task with 79 university students in China, we conducted two experiments with three- and four-character words to explore this issue. The results revealed clear character position processing with transposed pseudowords. Crucially, we identified a sequence effect in Chinese character position processing within words, directly supporting the hypothesis that character-based processing occurs with Chinese words. We also discussed other possibilities in Chinese character position processing.

Can faces affect object-based attention? Evidence from online experiments

This study tested how human faces affect object-based attention (OBA) through two online experiments in a modified double-rectangle paradigm. The results of Experiment 1 revealed that faces did not elicit the OBA effect as non-face objects, which was caused by a longer response time (RT) when attention is focused on faces relative to non-face objects. In addition, by observing faster RTs when attention was engaged horizontally rather than vertically, we found a significant horizontal attention bias, which might override the OBA effect if vertical rectangles were the only items presented; these results were replicated in Experiment 2 (using only vertical rectangles) after directly measuring horizontal bias and excluding its influence on the OBA effect. This study suggested that faces cannot elicit the same-object advantage in the double-rectangle paradigm and provided a method to measure the OBA effect free from horizontal bias.

Attention can operate on object representations in visual sensory memory

Numerous studies have shown that attention can be allocated to various types of objects, such as low-level objects developed by perceptual organization and high-level objects developed by semantic associations. However, little is known about whether attention can also be affected solely by object representations in the brain, after the disappearance of physical objects. Here, we used a modified double-rectangle paradigm to investigate how attention is affected by object representation in visual sensory memory when the physical objects disappear for a short period of time before the target onset. By manipulating the interstimulus interval (ISI) between the offset of the objects and the onset of the target, an object-based attention effect, with shorter reaction times (RTs) for within-object relative to between-object conditions, was observed in the short-ISI (within 500 ms in Experiments 1a, 1b, 2, and 3) conditions while disappearing in the long-ISI (800 ms in Experiment 4) conditions. This result demonstrated that the mere presence of object representation in visual sensory memory, or the sensory memory-maintained object, can serve as an object unit that attention can operate on. This provides evidence for the relationship between object-based attention and visual sensory memory: object representation in visual sensory memory could affect attentional allocation, or attention can operate on a sensory memory-maintained object.

Deafness enhances perceptual span size in Chinese reading: Evidence from a gaze-contingent moving-window paradigm

Using a gaze-contingent moving-window paradigm, we investigated whether/how deafness affects perceptual processing in Chinese reading. Besides the manipulation of window size, word length of sentences used in the experiment was also manipulated to check whether deafness enhanced the word length effect on perceptual span. Significant interactions of window constraints and deafness and a three-way interaction were observed on reading rate. Smaller effects of window constraints for deaf Chinese readers and nonreliable three-way interactions were observed on forward saccade length. This suggests that deaf Chinese readers exhibit a larger perceptual span, and word length affected the span from which information was acquired for comprehension whereas both deafness and word length might have little impact on the span from which information is acquired for oculomotor targeting during natural reading of Chinese.

A region complexity effect masquerading as object-based attention

A large portion of the evidence for object-based attention comes from experiments using the two-rectangle paradigm introduced by Egly, Driver, and Rafal (1994), in which response times are longer when the two stimulus locations relevant to the task are on separate objects. In the new experiments presented here, response times are longer when the two locations are part of the same object but are separated by a concavity in the object, so that the region directly between the two locations is crossed by the object's boundaries. Response times when the two locations are separated by the concavity are not statistically different from when they are on two separate objects. The results are similar for a two-letter comparison task and for a spatial cuing task. Thus, in these experiments, the response time increase does not reflect the cost of shifting attention from object to object, because it appears when the two locations are on the same object, and it is not increased when they are on different objects. Instead, it seems to reflect the complexity of the region between the two stimulus locations. This finding raises questions about whether data from previous two-rectangle experiments should be attributed to object-based attention.

Do visual word segmentation cues improve reading performance in Chinese reading？

It is controversial whether providing visual word segmentation cues can improve Chinese reading performance. This study investigated this topic by examining how visual word segmentation cues such as grey highlighting, red colour and interword spacing influence global sentence reading and local word recognition during reading Chinese text in three experiments. The results showed that interword spacing could facilitate local word recognition but could not increase reading speed. In contrast, grey highlighting and red colour could improve neither local word recognition nor global sentence reading performance. Instead, these cues increased the number of fixations and saccades, resulting in slower reading speed. These results suggest that even red colour is not a practically visual cue for Chinese word segmentation and the corresponding mechanisms were discussed. Practitioner Summary: We studied how visual cues such as grey highlighting, red colour and interword spacing influenced Chinese reading performance. Our data showed that even the red colour was not an efficient cue for Chinese word segmentation. The corresponding mechanisms and future direction were discussed regarding how to improve Chinese reading performance.

The impact of monetary stimuli on object-based attention

Previous research has implied that monetary reward to target location (a reward for spatial properties) can affect object-based attention, but no study has directly investigated the influence of monetary objects (a reward for object properties) on object-based attention. Thus, it is unclear whether and how monetary objects can affect object-based attention. To experimentally investigate this problem, this study adapted the well-established two-rectangle paradigm. In Experiment 1, either two 100-yuan notes or two 1-yuan notes were presented to participants. We found an object-based effect with faster responses to targets at an uncued position on the cued object compared to those at an equidistant position on the uncued object; the effect was similar in 100-yuan and 1-yuan note trials. In Experiment 2, two notes (one 100-yuan and one 1-yuan) were simultaneously presented to participants, and cue location (100-yuan, 1-yuan) was manipulated. We found a greater object-based effect when the cue appeared on the 100-yuan note than on the 1-yuan note. These results suggest that the rewarding property of objects can affect object-based attention by means of altering object salience.

Reinforcement Learning of Linking and Tracing Contours in Recurrent Neural Networks

The processing of a visual stimulus can be subdivided into a number of stages. Upon stimulus presentation there is an early phase of feedforward processing where the visual information is propagated from lower to higher visual areas for the extraction of basic and complex stimulus features. This is followed by a later phase where horizontal connections within areas and feedback connections from higher areas back to lower areas come into play. In this later phase, image elements that are behaviorally relevant are grouped by Gestalt grouping rules and are labeled in the cortex with enhanced neuronal activity (object-based attention in psychology). Recent neurophysiological studies revealed that reward-based learning influences these recurrent grouping processes, but it is not well understood how rewards train recurrent circuits for perceptual organization. This paper examines the mechanisms for reward-based learning of new grouping rules. We derive a learning rule that can explain how rewards influence the information flow through feedforward, horizontal and feedback connections. We illustrate the efficiency with two tasks that have been used to study the neuronal correlates of perceptual organization in early visual cortex. The first task is called contour-integration and demands the integration of collinear contour elements into an elongated curve. We show how reward-based learning causes an enhancement of the representation of the to-be-grouped elements at early levels of a recurrent neural network, just as is observed in the visual cortex of monkeys. The second task is curve-tracing where the aim is to determine the endpoint of an elongated curve composed of connected image elements. If trained with the new learning rule, neural networks learn to propagate enhanced activity over the curve, in accordance with neurophysiological data. We close the paper with a number of model predictions that can be tested in future neurophysiological and computational studies.

Our experience with the visual world allows us to group image elements that belong to the same perceptual object and to segregate them from other objects and the background. If subjects learn to group contour elements, this experience influences neuronal activity in early visual cortical areas, including the primary visual cortex (V1). Learning presumably depends on alterations in the pattern of connections within and between areas of the visual cortex. However, the processes that control changes in connectivity are not well understood. Here we present the first computational model that can train a neural network to integrate collinear contour elements into elongated curves and to trace a curve through the visual field. The new learning algorithm trains fully recurrent neural networks, provided the connectivity causes the networks to reach a stable state. The model reproduces the behavioral performance of monkeys trained in these tasks and explains the patterns of neuronal activity in the visual cortex that emerge during learning, which is remarkable because the only feedback for the model is a reward for successful trials. We discuss a number of the model predictions that can be tested in future neuroscientific work.

Strength of object representation: its key role in object-based attention for determining the competition result between Gestalt and top-down objects

It was found in previous studies that two types of objects (rectangles formed according to the Gestalt principle and Chinese words formed in a top-down fashion) can both induce an object-based effect. The aim of the present study was to investigate how the strength of an object representation affects the result of the competition between these two types of objects based on research carried out by Liu, Wang and Zhou [(2011) Acta Psychologica, 138(3), 397-404]. In Experiment 1, the rectangles were filled with two different colors to increase the strength of Gestalt object representation, and we found that the object effect changed significantly for the different stimulus types. Experiment 2 used Chinese words with various familiarities to manipulate the strength of the top-down object representation. As a result, the object-based effect induced by rectangles was observed only when the Chinese word familiarity was low. These results suggest that the strength of object representation determines the result of competition between different types of objects.

The effects of attention pre-allocation and target-background integration on object-based attention

Object-based attention has been documented as an important mechanism with which to control attention in several studies. To date, two main hypotheses have been proposed to interpret object-based attention: attention spreading and prioritization of search. There is evidence that supports these hypotheses in the literature. In the present study, we sought to compare these two hypotheses systematically by manipulating two factors: the integration of the target and background and the presence of attention pre-allocation. For this purpose, we used a flanker task in which the location of the task-relevant target was fixed, but the relationship between the target and the background varied. In addition, attention pre-allocation was presented in only half of the conditions. The results revealed that the attention spreading hypothesis was supported only when attention was not pre-allocated and target-background integration was high; however, the prioritization hypothesis was supported in all other conditions. Our findings provide insight into the comparisons of the attention spreading and prioritization hypotheses. Furthermore, our findings suggest that attention resources may be the underlying factor determining appropriate strategy in the control of attention.

How a face may affect object-based attention: evidence from adults and 8-month-old infants

Object-based attention operates on perceptual objects, opening the possibility that the costs and benefits humans have to pay to move attention between-objects might be affected by the nature of the stimuli. The current study reported two experiments with adults and 8-month-old infants investigating whether object-based-attention is affected by the type of stimulus (faces vs. non-faces stimuli). Using the well-known cueing task developed by Egly et al. (1994) to study the object-based component of attention, in Experiment 1 adult participants were presented with two upright, inverted or scrambled faces and an eye-tracker measured their saccadic latencies to find a target that could appear on the same object that was just cued or on the other object that was uncued. Data showed that an object-based effect (a smaller cost to shift attention within- compared to between-objects) occurred only with scrambled face, but not with upright or inverted faces. In Experiment 2 the same task was performed with 8-month-old infants, using upright and inverted faces. Data revealed that an object-based effect emerges only for inverted faces but not for upright faces. Overall, these findings suggest that object-based attention is modulated by the type of stimulus and by the experience acquired by the viewer with different objects.

Reading is fundamentally similar across disparate writing systems: a systematic characterization of how words and characters influence eye movements in Chinese reading

While much previous work on reading in languages with alphabetic scripts has suggested that reading is word-based, reading in Chinese has been argued to be less reliant on words. This is primarily because in the Chinese writing system words are not spatially segmented, and characters are themselves complex visual objects. Here, we present a systematic characterization of the effects of a wide range of word and character properties on eye movements in Chinese reading, using a set of mixed-effects regression models. The results reveal a rich pattern of effects of the properties of the current, previous, and next words on a range of reading measures, which is strikingly similar to the pattern of effects of word properties reported in spaced alphabetic languages. This finding provides evidence that reading shares a word-based core and may be fundamentally similar across languages with highly dissimilar scripts. We show that these findings are robust to the inclusion of character properties in the regression models and are equally reliable when dependent measures are defined in terms of characters rather than words, providing strong evidence that word properties have effects in Chinese reading above and beyond characters. This systematic characterization of the effects of word and character properties in Chinese advances our knowledge of the processes underlying reading and informs the future development of models of reading. More generally, however, this work suggests that differences in script may not alter the fundamental nature of reading.

Word properties of a fixated region affect outgoing saccade length in Chinese reading

In two experiments, we investigated how forward saccades are targeted in Chinese reading. In Experiment 1, the critical region was a 4-character string which was either a word (one-word condition) or two 2-character word phrases (two-word condition). In Experiment 2, the critical region was either a high frequency word or a low frequency word. The outgoing saccade length from the last fixation on the critical region was longer in the one-word condition than the two-word condition in Experiment 1 and was longer in the high frequency condition than in the low frequency condition in Experiment 2. These results indicate that the properties of words in a fixated region affect the length of the outgoing saccade. We propose a processing-based strategy for saccade target selection in Chinese reading in which readers estimate how many characters they can process on each fixation, and then program their next saccade so that the eyes fixate somewhere beyond them. As a consequence, the easier the processing of the fixated region is, the longer the outgoing saccade is.

Word boundaries affect visual attention in Chinese reading

In two experiments, we explored attention deployment during the reading of Chinese words using a probe detection task. In both experiments, Chinese readers saw four simplified Chinese characters briefly, and then a probe was presented at one of the character positions. The four characters constituted either one word or two words of two characters each. Reaction time was shorter when the probe was at the character 2 position than the character 3 position in the two-word condition, but not in the one-word condition. In Experiment 2, there were more trials and the materials were more carefully controlled, and the results replicated that of Experiment 1. These results suggest that word boundary information affects attentional deployment in Chinese reading.

How prevalent is object-based attention?

Previous research suggests that visual attention can be allocated to locations in space (space-based attention) and to objects (object-based attention). The cueing effects associated with space-based attention tend to be large and are found consistently across experiments. Object-based attention effects, however, are small and found less consistently across experiments. In three experiments we address the possibility that variability in object-based attention effects across studies reflects low incidence of such effects at the level of individual subjects. Experiment 1 measured space-based and object-based cueing effects for horizontal and vertical rectangles in 60 subjects comparing commonly used target detection and discrimination tasks. In Experiment 2 we ran another 120 subjects in a target discrimination task in which rectangle orientation varied between subjects. Using parametric statistical methods, we found object-based effects only for horizontal rectangles. Bootstrapping methods were used to measure effects in individual subjects. Significant space-based cueing effects were found in nearly all subjects in both experiments, across tasks and rectangle orientations. However, only a small number of subjects exhibited significant object-based cueing effects. Experiment 3 measured only object-based attention effects using another common paradigm and again, using bootstrapping, we found only a small number of subjects that exhibited significant object-based cueing effects. Our results show that object-based effects are more prevalent for horizontal rectangles, which is in accordance with the theory that attention may be allocated more easily along the horizontal meridian. The fact that so few individuals exhibit a significant object-based cueing effect presumably is why previous studies of this effect might have yielded inconsistent results. The results from the current study highlight the importance of considering individual subject data in addition to commonly used statistical methods.

Lexical- and perceptual-based object effects in the two-rectangle cueing paradigm

Previous studies demonstrate that attentional selection can be object-based, in which the object is defined in terms of Gestalt principles or lexical organizations. Here we investigate how attentional selection functions when the two types of objects are manipulated jointly. Experiment 1 replicated Li and Logan (2008) by showing that attentional shift between two Chinese characters is more efficient when they form a compound word than when they form a nonword. Experiment 2A presented characters either alone or within rectangles (Egly, Driver, & Rafal, 1994) and the characters in a rectangle formed either a word or a nonword. Experiment 2B differed from Experiment 2A in that the two characters forming a word were in different rectangles. Experiment 3A presented the two characters of a word either within a rectangle or in different rectangles. Experiment 3B used the same design as Experiment 3A but presented stimuli of different types in random orders, rather than in blocks as in Experiments 2A, 2B and 3A. In blocked presentation, detection responses to the target color on a character were faster when this character and the cue character formed a word than when they did not, and the size of this lexical-based object effect did not vary according to whether the two characters were presented alone or within or between rectangles. In random presentation, however, the lexical-based object effect was diminished when the two characters of a word were presented in different rectangles. Overall, these findings suggest that the processes that constrain attention deployment over conjoined objects can be strategically adjusted.

Eye movement guidance in Chinese reading: is there a preferred viewing location?

In this study, we examined eye movement guidance in Chinese reading. We embedded either a 2-character word or a 4-character word in the same sentence frame, and observed the eye movements of Chinese readers when they read these sentences. We found that when all saccades into the target words were considered that readers eyes tended to land near the beginning of the word. However, we also found that Chinese readers' eyes landed at the center of words when they made only a single fixation on a word, and that they landed at the beginning of a word when they made more than one fixation on a word. However, simulations that we carried out suggest that these findings cannot be taken to unambiguously argue for word-based saccade targeting in Chinese reading. We discuss alternative accounts of eye guidance in Chinese reading and suggest that eye movement target planning for Chinese readers might involve a combination of character-based and word-based targeting contingent on word segmentation processes.

Visual apparent motion can be modulated by task-irrelevant lexical information

Previous studies have repeatedly demonstrated the impact of Gestalt structural grouping principles upon the parsing of motion correspondence in ambiguous apparent motion. Here, by embedding Chinese characters in a visual Ternus display that comprised two stimulus frames, we showed that the perception of visual apparent motion can be modulated by activation of task-irrelevant lexical representations. Each frame had two disks, with the second disk of the first frame and the first disk of the second frame being presented at the same location. Observers could perceive either "element motion," in which the endmost disk is seen as moving back and forth while the middle disk at the central position remains stationary, or "group motion," in which both disks appear to move laterally as a whole. More reports of group motion, as opposed to element motion, were obtained when the embedded characters formed two-character compound words than when they formed nonwords, although this lexicality effect appeared to be attenuated by the use of the same characters at the overlapping position across the two frames. Thus, grouping of visual elements in a changing world can be guided by both structural principles and prior world knowledge, including lexical information.

Effects of instantaneous object input and past experience on object-based attention

Evidence for object-based attention typically comes from studies using displays with unchanged objects, and no consensus has yet been reached as to whether the object effect would be altered by changing object displays or having seen this change across-trials. We examined this by using modifications of the double-rectangle cuing paradigm of Egly et al. [Egly, R., Driver, J., & Rafal, R. D. (1994). Shifting visual attention between objects and locations: Evidence from normal and parietal lesion subjects. Journal of Experimental Psychology: General, 123, 161-177], and our results, when the objects remained unchanging, replicated the original object effect. However, no object effect was found when the rectangles disappeared from view in the last (target) frame. This was true regardless of the likelihood of the rectangles disappearing, indicating the importance of instantaneous object inputs for object-based attention. The across-trial experience of seeing a different object (boomerang), however, was found to influence the object effect when the cued rectangles persisted throughout the trial. Unlike previous studies, which emphasize one or the other, we demonstrate clearly that instantaneous object inputs and past experience interact to determine the way attention selects objects.