How does reading direction modulate perceptual asymmetry effects?

Understanding Human Cognition Through Computational Modeling

One important goal of cognitive science is to understand the mind in terms of its representational and computational capacities, where computational modeling plays an essential role in providing theoretical explanations and predictions of human behavior and mental phenomena. In my research, I have been using computational modeling, together with behavioral experiments and cognitive neuroscience methods, to investigate the information processing mechanisms underlying learning and visual cognition in terms of perceptual representation and attention strategy. In perceptual representation, I have used neural network models to understand how the split architecture in the human visual system influences visual cognition, and to examine perceptual representation development as the results of expertise. In attention strategy, I have developed the Eye Movement analysis with Hidden Markov Models method for quantifying eye movement pattern and consistency using both spatial and temporal information, which has led to novel findings across disciplines not discoverable using traditional methods. By integrating it with deep neural networks (DNN), I have developed DNN+HMM to account for eye movement strategy learning in human visual cognition. The understanding of the human mind through computational modeling also facilitates research on artificial intelligence's (AI) comparability with human cognition, which can in turn help explainable AI systems infer humans' belief on AI's operations and provide human-centered explanations to enhance human-AI interaction and mutual understanding. Together, these demonstrate the essential role of computational modeling methods in providing theoretical accounts of the human mind as well as its interaction with its environment and AI systems.

Adaptation under dichromatic illumination

Over the years, many CATs (chromatic adaptation transforms) have been developed, such as CMCCAT97, CAT02 and CAT16, to predict the corresponding colors under different illuminants. These CATs were derived from uniform simple stimuli surrounded by a uniform background with a single illuminant. Although some mixed adaptation models have been proposed in literature to predict the adaptation under more than one illuminant, these models are typically limited to a certain scene and exclude the impact of spatial complexity. To investigate chromatic adaptation under more complex conditions, an achromatic matching experiment was conducted with (simultaneously) spatially dichromatic illumination for three illumination color pairs and various spatial configurations. Spatial configuration was found to have an impact on both the degree of adaptation and the equivalent illuminant chromaticity, which is the chromaticity of a single uniform adapting illumination that results in the same corresponding colors as for the dichromatic lighting condition. A preliminary CAT model is proposed that considers the spatial and colorimetric complexity of the illumination.

Role of reading medium and audio distractors on visual search

PURPOSE

Visual search is an active perceptual task influenced by objective factors and subjective factors such as task difficulty, distractors, attention and familiarity respectively. We studied the effect of different search directions, task medium and presence or absence of audio distractors on visual search time in young normal subjects METHODS: Twenty-four young (19-27 years) subjects with normal ocular health (except refractive error) participated in the study after obtaining informed consent. Subjects performed a word search task of ten 7-letter words of medium difficulty level. It was performed by each subject in Up-down, Down-Up, Left-Right, Right-Left, Diagonal and Random directions, with equal number of distractors. The task was performed in paper and digital medium, with or without audio distractors. The conditions were performed in random order by each subject and the time taken to accurately complete the word search was documented for each condition.

RESULT

The visual search time (VST) was significantly different with different search directions (ANOVA p<0.0001, df=5), considering both digital and non-digital medium, with or without audio distractors. The average VST was the least for left-right search direction (100±7.2 s) and was highest for random search direction (291±19 s), on a digital medium (VST_digital: 183±77 s) and in presence of an audio distractor (VST_audio: 184±77 s). The VST scores were not correlated with the age (r=-0.14, p = 0.25).

CONCLUSION

The visual search time is significantly delayed for search direction other than left-right direction and in presence of an audio distractor on a digital medium. These factors could play a significant role in visual orientation and specific tasks such as reading.

Developmental trajectories of expert perception processing of Chinese characters in primary school children

Previous studies have demonstrated that inversion effect and left-side bias are stable expertise markers in Chinese character processing among adults. However, it is less clear how these markers develop early on (i.e., among primary school students). Therefore, this study aimed to investigate the development of the two markers by comparing primary school-aged students of three age groups (Grade 1, Grade 3, and Grade 5) and adults in tests of inversion effect (Experiment 1) and left-sided bias effect (Experiment 2). The results replicated that both effects during Chinese character processing were present among adults. However, more importantly, the effects were different among primary school-aged students in different grades: the inversion effect was found as early as in Grade 1, but the left-side bias effect did not emerge in Grade 1 and as approximated that of adults until Grade 3. The study suggested a potential dissociation in developing different aspects of expertise during Chinese character processing in early childhood.

Music reading experience modulates eye movement pattern in English reading but not in Chinese reading

Here we tested the hypothesis that in Chinese-English bilinguals, music reading experience may modulate eye movement planning in reading English but not Chinese sentences due to the similarity in perceptual demands on processing sequential symbol strings separated by spaces between music notation and English sentence reading. Chinese–English bilingual musicians and non-musicians read legal, semantically incorrect, and syntactically (and semantically) incorrect sentences in both English and Chinese. In English reading, musicians showed more dispersed eye movement patterns in reading syntactically incorrect sentences than legal sentences, whereas non-musicians did not. This effect was not observed in Chinese reading. Musicians also had shorter saccade lengths when viewing syntactically incorrect than correct musical notations and sentences in an unfamiliar alphabetic language (Tibetan), whereas non-musicians did not. Thus, musicians’ eye movement planning was disturbed by syntactic violations in both music and English reading but not in Chinese reading, and this effect was generalized to an unfamiliar alphabetic language. These results suggested that music reading experience may modulate perceptual processes in reading differentially in bilinguals’ two languages, depending on their processing similarities.

The Left-Side Bias Is Reduced to Other-Race Faces in Caucasian Individuals

One stable marker of face perception appears to be left-side bias, the tendency to rely more on information conveyed by the left side of the face than the right. Previous studies have shown that left-side bias is influenced by familiarity and prior experience with face stimuli. Since other-race facial recognition is characterized by reduced familiarity, in contrast to own-race facial recognition, the phenomenon of left-side bias is expected to be weaker for other-race faces. Among Chinese participants, face inversion has been found to eliminate the left-side bias associated with own-race faces. Therefore, it is of interest to know whether face inversion influences left-side bias for non-Chinese research participants and can be generalized across own- and other-race faces. This study assessed 65 Caucasian participants using upright and inverted chimeric Caucasian and Asian faces in an identity similarity-judgment task. Although a significant left-side bias was observed for upright own-race faces, this bias was eliminated by facial inversion, indicating that such a bias depends on the applicability of configural processing strategies. For other-race faces, there was no left-side bias in the upright condition. Interestingly, the inverted presentation yielded a right-side bias. These results show that while left-side bias is affected by familiarity differences between own- and other-race faces, it is a universal phenomenon for upright faces. Inverted presentation strongly reduces left-side bias and may even cause it to revert to right-side bias, suggesting that left-side bias depends on configural face processing.

Graphic complexity in writing systems

A writing system is a graphic code, i.e., a system of standardized pairings between symbols and meanings in which symbols take the form of images that can endure. The visual character of writing implies that written characters have to fit constraints of the human visual system. One aspect of this optimization lays in the graphic complexity of the characters used by scripts. Scripts are sets of graphic characters used for the written form of one language or more. Using computational methods over a large and diverse dataset (over 47,000 characters, from over 133 scripts), we answer three central questions about the visual complexity of written characters and the evolution of writing: (1) What determines character complexity? (2) Can we find traces of evolutionary change in character complexity? (3) Is complexity distributed in a way that makes character recognition easier? Our study suggests that (1) character complexity depends primarily on which linguistic unit the characters encode, and that (2) there is little evidence of evolutionary change in character complexity. Additionally (3) for an individual character, the half which is encountered first while reading tends to be more complex than that which is encountered last.

The depth and breadth of multiple perceptual asymmetries in right handers and non-right handers

Several non-verbal perceptual and attentional processes have been linked with specialization of the right cerebral hemisphere. Given that most people have a left hemispheric specialization for language, it is tempting to assume that functions of these two classes of dominance are related. Unfortunately, such models of complementarity are notoriously hard to test. Here we suggest a method which compares frequency of a particular perceptual asymmetry with known frequencies of left hemispheric language dominance in right-handed and non-right handed groups. We illustrate this idea using the greyscales and colourscales tasks, chimeric faces, emotional dichotic listening, and a consonant-vowel dichotic listening task. Results show a substantial "breadth" of leftward bias on the right hemispheric tasks and rightward bias on verbal dichotic listening. Right handers and non-right handers did not differ in terms of proportions of people who were left biased for greyscales/colourscales. Support for reduced typical biases in non-right handers was found for chimeric faces and for CV dichotic listening. Results are discussed in terms of complementary theories of cerebral asymmetries, and how this type of method could be used to create a taxonomy of lateralized functions, each categorized as related to speech and language dominance, or not.

Left-Side Bias Is Observed in Sequential Matching Paradigm for Face Processing

The left-side bias refers to how a chimeric face is created from the left side of a face (from the viewer’s perspective) and its mirror image are considered more similar to the original face than a chimeric face created from the right side of the same face and its mirror image. Previous studies investigated the left-side bias by using the chimeric stimuli task, where the original face and chimeric face were presented simultaneously. However, it remains unclear whether left-side bias effect is observed when the original face and chimeric face are presented sequentially. We completed two experiments using the sequential matching paradigm to investigate this issue. The results from both Experiment 1 and 2 showed that participants judged the identical proportion of the left chimeric face and original face was significantly higher than that of the right chimeric face and original face, which implies that the left-side bias effect can be observed in the sequential matching paradigm for face processing.

The Presentation Location of the Reference Stimuli Affects the Left-Side Bias in the Processing of Faces and Chinese Characters

For faces and Chinese characters, a left-side processing bias, in which observers rely more heavily on information conveyed by the left side of stimuli than the right side of stimuli, has been frequently reported in previous studies. However, it remains unclear whether this left-side bias effect is modulated by the reference stimuli's location. The present study adopted the chimeric stimuli task to investigate the influence of the presentation location of the reference stimuli on the left-side bias in face and Chinese character processing. The results demonstrated that when a reference face was presented in the left visual field of its chimeric images, which are centrally presented, the participants showed a preference higher than the no-bias threshold for the left chimeric face; this effect, however, was not observed in the right visual field. This finding indicates that the left-side bias effect in face processing is stronger when the reference face is in the left visual field. In contrast, the left-side bias was observed in Chinese character processing when the reference Chinese character was presented in either the left or right visual field. Together, these findings suggest that although faces and Chinese characters both have a left-side processing bias, the underlying neural mechanisms of this left-side bias might be different.