The impact of presentation modes on mental rotation processing: a comparative analysis of eye movements and performance

Mental rotation is the ability to rotate mental representations of objects in space. Shepard and Metzler's shape-matching tasks, frequently used to test mental rotation, involve presenting pictorial representations of 3D objects. This stimulus material has raised questions regarding the ecological validity of the test for mental rotation with actual visual 3D objects. To systematically investigate differences in mental rotation with pictorial and visual stimuli, we compared data of N = 54 university students from a virtual reality experiment. Comparing both conditions within subjects, we found higher accuracy and faster reaction times for 3D visual figures. We expected eye tracking to reveal differences in participants' stimulus processing and mental rotation strategies induced by the visual differences. We statistically compared fixations (locations), saccades (directions), pupil changes, and head movements. Supplementary Shapley values of a Gradient Boosting Decision Tree algorithm were analyzed, which correctly classified the two conditions using eye and head movements. The results indicated that with visual 3D figures, the encoding of spatial information was less demanding, and participants may have used egocentric transformations and perspective changes. Moreover, participants showed eye movements associated with more holistic processing for visual 3D figures and more piecemeal processing for pictorial 2D figures.

Slower but more accurate mental rotation performance in aphantasia linked to differences in cognitive strategies

Mental rotation tasks are frequently used as standard measures of mental imagery. However, aphantasia research has brought such use into question. Here, we assessed a large group of individuals who lack visual imagery (aphantasia) on two mental rotation tasks: a three-dimensional block-shape, and a human manikin rotation task. In both tasks, those with aphantasia had slower, but more accurate responses than controls. Both groups demonstrated classic linear increases in response time and error-rate as functions of angular disparity. In the three-dimensional block-shape rotation task, a within-group speed-accuracy trade-off was found in controls, whereas faster individuals in the aphantasia group were also more accurate. Control participants generally favoured using object-based mental rotation strategies, whereas those with aphantasia favoured analytic strategies. These results suggest that visual imagery is not crucial for successful performance in classical mental rotation tasks, as alternative strategies can be effectively utilised in the absence of holistic mental representations.

Cognitive processing stages in mental rotation - How can cognitive modelling inform HsMM-EEG models?

The aspiration for insight into human cognitive processing has traditionally driven research in cognitive science. With methods such as the Hidden semi-Markov Model-Electroencephalography (HsMM-EEG) method, new approaches have been developed that help to understand the temporal structure of cognition by identifying temporally discrete processing stages. However, it remains challenging to assign concrete functional contributions by specific processing stages to the overall cognitive process. In this paper, we address this challenge by linking HsMM-EEG3 with cognitive modelling, with the aim of further validating the HsMM-EEG3 method and demonstrating the potential of cognitive models to facilitate functional interpretation of processing stages. For this purpose, we applied HsMM-EEG3 to data from a mental rotation task and developed an ACT-R cognitive model that is able to closely replicate human performance in this task. Applying HsMM-EEG3 to the mental rotation experiment data revealed a strong likelihood for 6 distinct stages of cognitive processing during trials, with an additional stage for non-rotated conditions. The cognitive model predicted intra-trial mental activity patterns that project well onto the processing stages, while explaining the additional stage as a marker of non-spatial shortcut use. Thereby, this combined methodology provided substantially more information than either method by itself and suggests conclusions for cognitive processing in general.

Eye movement characteristics in a mental rotation task presented in virtual reality

Introduction

Eye-tracking technology provides a reliable and cost-effective approach to characterize mental representation according to specific patterns. Mental rotation tasks, referring to the mental representation and transformation of visual information, have been widely used to examine visuospatial ability. In these tasks, participants visually perceive three-dimensional (3D) objects and mentally rotate them until they identify whether the paired objects are identical or mirrored. In most studies, 3D objects are presented using two-dimensional (2D) images on a computer screen. Currently, visual neuroscience tends to investigate visual behavior responding to naturalistic stimuli rather than image stimuli. Virtual reality (VR) is an emerging technology used to provide naturalistic stimuli, allowing the investigation of behavioral features in an immersive environment similar to the real world. However, mental rotation tasks using 3D objects in immersive VR have been rarely reported.

Methods

Here, we designed a VR mental rotation task using 3D stimuli presented in a head-mounted display (HMD). An eye tracker incorporated into the HMD was used to examine eye movement characteristics during the task synchronically. The stimuli were virtual paired objects oriented at specific angular disparities (0, 60, 120, and 180°). We recruited thirty-three participants who were required to determine whether the paired 3D objects were identical or mirrored.

Results

Behavioral results demonstrated that the response times when comparing mirrored objects were longer than identical objects. Eye-movement results showed that the percent fixation time, the number of within-object fixations, and the number of saccades for the mirrored objects were significantly lower than that for the identical objects, providing further explanations for the behavioral results.

Discussion

In the present work, we examined behavioral and eye movement characteristics during a VR mental rotation task using 3D stimuli. Significant differences were observed in response times and eye movement metrics between identical and mirrored objects. The eye movement data provided further explanation for the behavioral results in the VR mental rotation task.

Video Game Play Does Not Improve Spatial Skills When Controlling for Speed-Accuracy Trade-Off: Evidence From Mental-Rotation and Mental-Folding Tasks

Researchers have been divided on the efficacy of computerized cognitive training (CCT) for enhancing spatial abilities, transfer of training, and improving malleability of skills. In this study, we assessed the effects of puzzle video game training on subsequent mental rotation (MR) and mental folding (MF) performance among adults with no cognitive impairment. We assessed participants at baseline with the Shepard-Metzler MR test followed by the differential aptitude test: space relations MF test (i.e., far transfer). We ranked participants' skills on these pre-tests and used a matching technique to form two skill groups from which we then randomly assigned members of each skill group either to an experimental group or a wait-list control group. The experimental group played two puzzle video games closely related to two-dimensional and three-dimensional MR tasks during 4-week training sessions (total of 12 hour of video games). Post-training, participants completed the MR and MF tests again. Two months later, we re-assessed only the experimental group's spatial skills to explore the sustainability of the trained performance. In addition to response times (RT) and error scores (ES), reported separately, we combined these variables into rate correct scores (RCS) to form an integrated measure of potential speed-accuracy trade-offs (SAT). As a result, we did not find significant improvements in MR performance from CCT engagement, nor did participants show a transfer of skills obtained by practicing MR-related puzzle games to a MF task. Based on the current findings, we urge caution when proposing a game-based intervention as a training tool to enhance spatial abilities. We argue that separately interpreting individual test measures can be misleading, as they only partially represent performance. In contrast, composite scores illuminate underlying cognitive strategies and best determine whether an observed improvement is attributable to enhanced capacities or individual heuristics and learned cognitive shortcuts.

The effect of autistic traits on disembedding and mental rotation in neurotypical women and men

Recent data has revealed dissociations between social and non-social skills in both autistic and neurotypical populations. In the present study, we investigated whether specific visuospatial abilities, such as figure disembedding and mental rotation, are differently related to social and non-social autistic traits, in neurotypical women and men. University students (N = 426) completed the Autism Spectrum Quotient (AQ), figure disembedding and mental rotation of two-dimensional figures tasks. AQ social skills (AQ-social) and attention-to-details (AQ-attention) subscales were used as measures of social and non-social autistic traits, respectively. Mental rotation was affected by a significant interaction between sex, social and non-social traits. When non-social traits were above the mean (+ 1 SD), no sex differences in mental rotation were found. Instead, below this value, sex differences depended on the social traits, with men on average outperforming women at middle-to-high social traits, and with a comparable performance, and with women on average outperforming men, at lower social traits. A small positive correlation between figure disembedding and social traits was observed in the overall sample. These results are interpreted in terms of the hyper-systemizing theory of autism and contribute to the evidence of individual differences in the cognitive style of autistic people and neurotypical people with autistic traits.

Perceived and mentally rotated contents are differentially represented in cortical depth of V1

Primary visual cortex (V1) in humans is known to represent both veridically perceived external input and internally-generated contents underlying imagery and mental rotation. However, it is unknown how the brain keeps these contents separate thus avoiding a mixture of the perceived and the imagined which could lead to potentially detrimental consequences. Inspired by neuroanatomical studies showing that feedforward and feedback connections in V1 terminate in different cortical layers, we hypothesized that this anatomical compartmentalization underlies functional segregation of external and internally-generated visual contents, respectively. We used high-resolution layer-specific fMRI to test this hypothesis in a mental rotation task. We found that rotated contents were predominant at outer cortical depth bins (i.e. superficial and deep). At the same time perceived contents were represented stronger at the middle cortical bin. These results identify how through cortical depth compartmentalization V1 functionally segregates rather than confuses external from internally-generated visual contents. These results indicate that feedforward and feedback manifest in distinct subdivisions of the early visual cortex, thereby reflecting a general strategy for implementing multiple cognitive functions within a single brain region.

Two people, one graph: the effect of rotated viewpoints on accessibility of data visualizations

In co-located, multi-user settings such as multi-touch tables, user interfaces need to be accessible from multiple viewpoints. In this project, we investigated how this goal can be achieved for depictions of data in bar graphs. We designed a laboratory task in which participants answered simple questions based on information depicted in bar graphs presented from differently rotated points of view. As the dependent variable, we measured differences in response onsets relative to the standard viewpoint (i.e., upright graphs). In Experiment 1, we manipulated graph and label orientation independently of each other. We observed that rotations of the labels rather than rotations of the graph itself pose a challenge for accessing depicted information from rotated viewpoints. In Experiment 2, we studied whether replacing word labels with pictographs could overcome the detrimental effects of rotated labels. Rotated pictographs were less detrimental than rotated word labels, but performance was still worse than in the unrotated baseline condition. In Experiment 3, we studied whether color coding could overcome the detrimental effects of rotated labels. Indeed, for multicolored labels, the detrimental effect of label rotation was in the negligible range. We discuss the implications of our findings for the underlying psychological theory as well as for the design of depicted statistical information in multi-user settings.

Identifying solution strategies in a mentalrotation test with gender-stereotyped objects

Many studies deal with solution strategies in mental-rotation tests. The approaches range from global analysis, attention to object parts, holistic and piecemeal strategy to a combined strategy. Other studies do not speak of strategies, but of holistic or piecemeal processes or even of holistic or piecemeal rotation. The methodological approach used here is to identify mental-rotation strategies via gaze patterns derived from eye-tracking data when solving chronometric mental-rotation tasks with gender-stereotyped objects. The mental-rotation test consists of 3 male-stereotyped objects (locomotive, hammer, wrench) and 3 femalestereotyped objects (pram, hand mirror, brush) rotated at eight different angles. The sample consisted of 16 women and 10 men (age: M=21.58; SD=4.21). The results of a qualitative analysis with two individual objects (wrench and brush) showed four different gaze patterns. These gaze patterns appeared with different frequency in the two objects and correlated differently with performance and response time. The results indicate either an objectoriented or an egocentric mental-rotation strategy behind the gaze patterns. In general, a new methodological approach has been developed to identify mental-rotation strategies bottom-up which can also be used for other stimulus types.

Mental Rotation: The Effects of Processing Strategy, Gender and Task Characteristics on Children's Accuracy, Reaction Time and Eye Movements' Pattern

The effects of gender, strategy and task characteristics on children's mental rotation (MR) behavioral measures and eye movements were studied. Eye movements reflect thinking pattern and assist understanding mental rotation performance. Eighty-three fourth-grade children (44 boys and 39 girls) were administered the Computerized Windows Mental Rotation test (CWMR) while having their eye movements monitored and completed a Strategy Self-Report (global/local/combined) and a Spatial Span (WM) subtest. Difficulty level affected performance and was reflected in a different eye movement pattern. Boys were more accurate than girls, but they did not differ in their eye movement pattern. Eye movement pattern was related to strategy, accu-racy and reaction time, revealing that the global and combined strategy were more effective compared with local strategy. WM was found to correlate with accuracy at the easy level of the test. The usage of eye movement measures assists in elaborating our knowledge regarding MR performance among chil-dren and enable a wider understanding regarding the interaction between gender, strategy and difficulty level.

Which properties of the visual stimuli predict the type of representation used in mental rotation?

Two modes of internal representation, holistic and piecemeal transformation, have been reported as a means to perform mental rotation (MR) tasks. The stimulus complexity effect has been proposed as an indicator to disentangle between these two representation types. However, the complexity effect has not been fully confirmed owing to the fact that different performances could result from different types of stimuli. Moreover, whether the non-mirror foils play a role in forcing participants to encode all the information from the stimuli in MR tasks is still under debate. This study aims at testing the association between these two common types of representation with different stimuli in MR tasks. First, the numbers of segments and vertices in polygon stimuli were manipulated to test which property of the visual stimuli is more likely to influence the representation in MR tasks. Second, the role of non-mirror foils was examined by comparing the stimulus complexity effect in both with- and without-non-mirror foils conditions. The results revealed that the segment number affected the slope of the linear function relating response times to rotation angle, but the vertex number in the polygons did not. This suggests that a holistic representation was more likely to be adopted in processing integrated objects, whereas a piecemeal transformation was at play in processing multi-part objects. In addition, the stimulus complexity effect was observed in the with-non-mirror foils condition but not in the without-non-mirror foils one, providing a direct evidence to support the role of non-mirror foils in MR tasks.

Differential influence of habitual third-person vision of a body part on mental rotation of images of hands and feet

Left/right judgement (LRJ) tasks involve determining the laterality of presented hand or feet images. Allocentric images (third-person perspective; 3PP) take longer to identify than egocentric images (first-person perspective; 1PP), supporting that implicit motor imagery (IMI)-mentally manoeuvring one's body to match the shown posture-is used. While numerous cognitive processes are involved during LRJs, it remains unclear whether features of the individual (e.g., visual exposure, experience, task-dependent use) influence the type of recognition strategy used during LRJs (IMI versus non-IMI). To investigate whether an individual's routine visual exposure to hands/feet in 3PP disrupts the typical perspective-reaction time (RT) relationship in LRJs, hand therapists, podiatrists, and healthy controls completed online LRJ tasks of hand and feet images. A group-specific reduction in RT for only allocentric images would represent a switch to non-IMI strategies. The results show that routine visual exposure to feet in 3PP (podiatrists) results in quicker RTs only for allocentric images of feet, suggesting a switch from IMI to non-IMI (e.g., visual object-based recognition) strategies. In contrast, routine visual exposure to hands in 3PP (hand therapists) does not alter RT for allocentric images, suggesting maintenance of IMI. However, hand therapists have quicker RTs (vs other groups) for egocentric hand images, supporting enhanced sensorimotor processing for the hand, consistent with task-dependent use (precise hand use). Higher accuracy in health professionals (vs control) on both tasks supports enhanced body schema. Combined, this suggests that 3PP visual exposure to body parts and task-dependent use contribute to LRJ performance/recognition strategy.

Age differences in spatial memory for mediated environments

Compared with younger adults, older adults have more difficulty with navigation and spatial memory in both familiar and unfamiliar domains. However, the cognitive mechanisms underlying these effects have been little explored. We examined three potential factors: (a) use of and coordination across spatial reference frames, (b) nonspatial cognitive abilities, and (c) the ability to segment a route into effective chunks. In two experiments, healthy young and older adults watched videos of navigation in a novel environment and had to remember the placement of landmarks along the route. Participants completed three spatial memory tasks-a virtual pointing task, a distance estimation task, and sketch map drawing-for each route. The pointing task depends on updating and accessing the updated egocentric reference frame relative to other frames. Map drawing may rely more on environment-centered processing. The distance estimation task could be solved using either frame of reference. Last, participants segmented each route. In a separate session, working memory, processing speed, and verbal memory were assessed. Older adults performed less well on all spatial tasks compared with younger adults; aging had a stronger negative effect on pointing performance. This may point to impairments in older adults' ability to update and coordinate information across reference frames. Performance on all spatial tasks was predicted by nonspatial task performance. Segmentation did not predict spatial memory. These results underline the importance of situating age differences in navigation in the context of basic transformations of spatial reference frames, and also in the context of nonspatial cognitive abilities. (PsycINFO Database Record

Different representations and strategies in mental rotation

It is still debated whether holistic or piecemeal transformation is applied to carry out mental rotation (MR) as an aspect of visual imagery. It has been recently argued that various mental representations could be flexibly generated to perform MR tasks. To test the hypothesis that imagery ability and types of stimuli interact to affect the format of representation and the choice of strategy in performing MR task, participants, grouped as good or poor imagers, were assessed using four MR tasks, comprising two sets of 'Standard' cube figures and two sets of 'non-Standard' ones, designed by withdrawing cubes from the Standard ones. Both good and poor imagers performed similarly under the two Standard conditions. Under non-Standard conditions, good imagers performed much faster in non-Standard objects than Standard ones, whereas poor imagers performed much slower in non-Standard objects than Standard ones. These results suggested that (1) individuals did not differ in processing the integrated Standard object, whereas (2) in processing the non-Standard objects, various visual representations and strategies could be applied in MR by diverse individuals: Good imagers were more flexible in generating different visual representations, whereas poor imagers applied different strategies under different task demands.

Uncovering the cognitive processes underlying mental rotation: an eye-movement study

Mental rotation is an important paradigm for spatial ability. Mental-rotation tasks are assumed to involve five or three sequential cognitive-processing states, though this has not been demonstrated experimentally. Here, we investigated how processing states alternate during mental-rotation tasks. Inference was carried out using an advanced statistical modelling and data-driven approach - a discriminative hidden Markov model (dHMM) trained using eye-movement data obtained from an experiment consisting of two different strategies: (I) mentally rotate the right-side figure to be aligned with the left-side figure and (II) mentally rotate the left-side figure to be aligned with the right-side figure. Eye movements were found to contain the necessary information for determining the processing strategy, and the dHMM that best fit our data segmented the mental-rotation process into three hidden states, which we termed encoding and searching, comparison, and searching on one-side pair. Additionally, we applied three classification methods, logistic regression, support vector model and dHMM, of which dHMM predicted the strategies with the highest accuracy (76.8%). Our study did confirm that there are differences in processing states between these two of mental-rotation strategies, and were consistent with the previous suggestion that mental rotation is discrete process that is accomplished in a piecemeal fashion.

An fMRI Study of the Impact of Block Building and Board Games on Spatial Ability

Previous studies have found that block play, board games, and puzzles result in better spatial ability. This study focused on examining the differential impact of structured block play and board games on spatial processing. Two groups of 8-year-old children were studied. One group participated in a five session block play training paradigm and the second group had a similar training protocol but played a word/spelling board game. A mental rotation task was assessed before and after training. The mental rotation task was performed during fMRI to observe the neural changes associated with the two play protocols. Only the block play group showed effects of training for both behavioral measures and fMRI measured brain activation. Behaviorally, the block play group showed improvements in both reaction time and accuracy. Additionally, the block play group showed increased involvement of regions that have been linked to spatial working memory and spatial processing after training. The board game group showed non-significant improvements in mental rotation performance, likely related to practice effects, and no training related brain activation differences. While the current study is preliminary, it does suggest that different “spatial” play activities have differential impacts on spatial processing with structured block play but not board games showing a significant impact on mental rotation performance.

Capacity for Visual Features in Mental Rotation

Although mental rotation is a core component of scientific reasoning, little is known about its underlying mechanisms. For instance, how much visual information can someone rotate at once? We asked participants to rotate a simple multipart shape, requiring them to maintain attachments between features and moving parts. The capacity of this aspect of mental rotation was strikingly low: Only one feature could remain attached to one part. Behavioral and eye-tracking data showed that this single feature remained "glued" via a singular focus of attention, typically on the object's top. We argue that the architecture of the human visual system is not suited for keeping multiple features attached to multiple parts during mental rotation. Such measurement of capacity limits may prove to be a critical step in dissecting the suite of visuospatial tools involved in mental rotation, leading to insights for improvement of pedagogy in science-education contexts.

Mental rotation and the motor system: embodiment head over heels

We examined whether body parts attached to abstract stimuli automatically force embodiment in a mental rotation task. In Experiment 1, standard cube combinations reflecting a human pose were added with (1) body parts on anatomically possible locations, (2) body parts on anatomically impossible locations, (3) colored end cubes, and (4) simple end cubes. Participants (N=30) had to decide whether two simultaneously presented stimuli, rotated in the picture plane, were identical or not. They were fastest and made less errors in the possible-body condition, but were slowest and least accurate in the impossible-body condition. A second experiment (N=32) replicated the results and ruled out that the poor performance in the impossible-body condition was due to the specific stimulus material. The findings of both experiments suggest that body parts automatically trigger embodiment, even when it is counterproductive and dramatically impairs performance, as in the impossible-body condition. It can furthermore be concluded that body parts cannot be used flexibly for spatial orientation in mental rotation tasks, compared to colored end cubes. Thus, embodiment appears to be a strong and inflexible mechanism that may, under certain conditions, even impede performance.