Spatial attention based on 2D location and relative depth order modulates visual working memory in a 3D environment

The influence of depth on object selection and manipulation in visual working memory within a 3D context

Recent studies have examined whether the internal selection mechanism functions similarly for perception and visual working memory (VWM). However, the process of how we access and manipulate object representations distributed in a 3D space remains unclear. In this study, we utilized a memory search task to investigate the effect of depth on object selection and manipulation within VWM. The memory display consisted of colored items half positioned at the near depth plane and the other half at the far plane. During memory maintenance, the participants were instructed to search for a target representation and update its color. The results showed that under object-based attention (Experiments 1, 3, and 5), the update time was faster for targets at the near plane than for those at the far plane. This effect was absent in VWM when deploying spatial attention (Experiment 2) and in visual search regardless of the type of attention deployed (Experiment 4). The differential effects of depth on spatial and object-based attention in VWM suggest that spatial attention primarily relied on 2D location information irrespective of depth, whereas object-based attention seemed to prioritize memory representations at the front plane before shifting to the back. Our findings shed light on the interaction between depth perception and the selection mechanisms within VWM in a 3D context, emphasizing the importance of ordinal, rather than metric, spatial information in guiding object-based attention in VWM.

Top-down modulation on depth processing: Visual searches for metric and ordinal depth information show a pattern of dissociation

Depending on the goal, one can selectively process the metric depth or the ordinal depth information in the same scene. It is unknown whether the metric depth and ordinal depth information are processed through a shared or different underlying mechanisms. Here, we investigated the processing of the metric depth and ordinal depth using visual search. Items were presented at multiple depth planes defined by the binocular disparity, with one item per depth plane. In the metric-search task, participants were required to search for the target on a particular depth plane, among one to three distractors. In the ordinal-search task, the target was specified by its depth order indicated by numbers (smaller numbers indicated nearer depth planes). We found that the ordinal search was faster and more accurate than the metric search, and the data showed a pattern of dissociation. Metric search, but not ordinal search, was slowed when the target and distractors were closer in depth, while ordinal search was slower for the middle than the edge positions but metric search was unaffected. These two opposite effects suggest that metric depth and ordinal depth may be processed differently.