Fixating picture boundaries does not eliminate boundary extension: implications for scene representation

Setting the scene for boundary extension: Methods, findings, connections, and theories

A previously viewed scene is often remembered as containing a larger extent of the background than was actually present, and information that was likely present just outside the boundaries of that view is often incorporated into the representation of that scene. This has been referred to as boundary extension. Methodologies used in studies on boundary extension (terminology, stimulus presentation, response measures) are described. Empirical findings regarding effects of characteristics of the stimulus (whether the stimulus depicts a scene, semantics of the scene, view angle, object size, object cropping, object orientation, object color, number of objects, depth of field, object distance, viewpoint production, scene orientation, motion, faces, emotions, modality, whether the scene is multimodal), characteristics of the display (aperture shape, aperture size, target duration, retention interval), and characteristics of the observer (allocation of attention, imagination, age, expectations and strategies, eye fixation, eye movements, monocular or binocular view, vantage point, confinement, prior exposure, expertise, arousal, pathology) on boundary extension are reviewed. Connections of boundary extension to other cognitive phenomena and processes (evolutionary adaptation, Gestalt principles, illusions, psychophysics, invariant physical principles, aesthetics, temporal boundary extension, normalization) are noted, and theories and theoretical considerations regarding boundary extension (multisource model, boundary transformation, mental imagery, 4E cognition, cognitive modularity, neurological mechanisms of scene representation) are discussed.

Object-based attention during scene perception elicits boundary contraction in memory

Boundary contraction and extension are two types of scene transformations that occur in memory. In extension, viewers extrapolate information beyond the edges of the image, whereas in contraction, viewers forget information near the edges. Recent work suggests that image composition influences the direction and magnitude of boundary transformation. We hypothesize that selective attention at encoding is an important driver of boundary transformation effects, selective attention to specific objects at encoding leading to boundary contraction. In this study, one group of participants (N = 36) memorized 15 scenes while searching for targets, while a separate group (N = 36) just memorized the scenes. Both groups then drew the scenes from memory with as much object and spatial detail as they could remember. We asked online workers to provide ratings of boundary transformations in the drawings, as well as how many objects they contained and the precision of remembered object size and location. We found that search condition drawings showed significantly greater boundary contraction than drawings of the same scenes in the memorize condition. Search drawings were significantly more likely to contain target objects, and the likelihood to recall other objects in the scene decreased as a function of their distance from the target. These findings suggest that selective attention to a specific object due to a search task at encoding will lead to significant boundary contraction.

Reduced peripheral vision in glaucoma and boundary extension

CLINICAL RELEVANCE

Peripheral vision is known to be critical for spatial navigation. However, visual cognition, which impacts peripheral vision, has not been studied extensively in glaucoma.

BACKGROUND

Spatial memory was assessed with a known to induce a robust memory distortion called "boundary extension" in which participants erroneously remember seeing more of a scene than was present in the sensory input.

METHODS

Fifteen patients with glaucoma and 15 age-matched normally sighted controls participated in the experiment. Participants were shown 10 photographs of natural scenes randomly displayed for 0.5 s or 10 s. Following each scene, the participant was asked to draw it from memory.

RESULTS

On average, boundary extension was larger, by 12%, for patients than for controls, but the difference was significant for 4 photographs. Patients tended to add more space between the object and the edges than there was between the objects and the border of the photograph. A control experiment in which participants were asked to draw isolated objects without scene context resulted in a significant reduction of the memory distortion in both groups, but patients still drew the objects smaller than controls.

CONCLUSION

The reduced field of view in glaucoma has an impact on spatial memory for scenes and on perception of size.

Does Boundary Extension Need Attention?

When we look at a picture, we tend to remember it by enriching the constructed mental representation with elements not present but probable outside the current view. The tendency to remember the perceived view with a broader scope is known as boundary extension (BE). Does BE benefit from paying reduced attention to the picture? While attention plays a central role in memory, only a few studies to date have investigated this question in the field of BE. In this research, participants completed a BE task in single- and dual-task conditions. The results indicate that BE is eliminated when the attention is divided on the onset of scene construction. We therefore discuss the role of attention in BE.

Perceived Distance Alters Memory for Scene Boundaries

Memory often fills in what is not there. A striking example of this is boundary extension, whereby observers mistakenly recall a view that extends beyond what was seen. However, not all visual memories extend in this way, which suggests that this process depends on specific scene properties. What factors determine when visual memories will include details that go beyond perceptual experience? Here, seven experiments (N = 1,100 adults) explored whether spatial scale-specifically, perceived viewing distance-drives boundary extension. We created fake miniatures by exploiting tilt shift, a photographic effect that selectively reduces perceived distance while preserving other scene properties (e.g., making a distant railway appear like a model train). Fake miniaturization increased boundary extension for otherwise identical scenes: Participants who performed a scene-memory task misremembered fake-miniaturized views as farther away than they actually were. This effect went beyond low-level image changes and generalized to a completely different distance manipulation. Thus, visual memory is modulated by the spatial scale at which the environment is viewed.

Effect of contextual knowledge on spatial layout extrapolation

Boundary extension (BE) refers to the tendency to remember a previously perceived scene with a greater spatial expanse. This phenomenon is described as resulting from different sources of information: external (i.e., visual) and internally driven (i.e., amodal, conceptual, and contextual) information. Although the literature has emphasized the role of top-down expectations to account for layout extrapolation, their effect has rarely been tested experimentally. In this research, we attempted to determine how visual context affects BE, as a function of scene exposure duration (long, short). To induce knowledge about visual context, the memorization phase of the camera distance paradigm was preceded by a preexposure phase, during which each of the to-be-memorized scenes was presented in a larger spatial framework. In an initial experiment, we examined the effect of contextual knowledge with presentation duration, allowing for in-depth processing of visual information during encoding (i.e., 15 s). The results indicated that participants exposed to the preexposure showed decreased BE, and displayed no directional memory error in some conditions. Because the effect of context is known to occur at an early stage of scene perception, in a second experiment we sought to determine whether the effect of a preview occurs during the first fixation on a visual scene. The results indicated that BE seems not to be modulated by this factor at very brief presentation durations. These results are discussed in light of current visual scene representation theories.

Boundary Extension in Face Processing

Boundary extension is a common false memory error, in which people confidently remember seeing a wider angle view of the scene than was viewed. Previous research found that boundary extension is scene-specific and did not examine this phenomenon in nonscenes. The present research explored boundary extension in cropped face images. Participants completed either a short-term or a long-term condition of the task. During the encoding, they observed photographs of faces, cropped either in a forehead or in a chin area, and subsequently performed face recognition through a forced-choice selection. The recognition options represented different degrees of boundary extension and boundary restriction errors. Eye-tracking and performance data were collected. The results demonstrated boundary extension in both memory conditions. Furthermore, previous literature reported the asymmetry in amounts of expansion at different sides of an image. The present work provides the evidence of asymmetry in boundary extension. In the short-term condition, boundary extension errors were more pronounced for forehead, than for chin face areas. Finally, this research examined the relationships between the measures of boundary extension, imagery, and emotion. The results suggest that individual differences in emotional ability and object, but not spatial, imagery could be associated with boundary extension in face processing.

Anticipatory scene representation in preschool children's recall and recognition memory

Behavioral and neuroscience research on boundary extension (false memory beyond the edges of a view of a scene) has provided new insights into the constructive nature of scene representation, and motivates questions about development. Early research with children (as young as 6-7 years) was consistent with boundary extension, but relied on an analysis of spatial errors in drawings which are open to alternative explanations (e.g. drawing ability). Experiment 1 replicated and extended prior drawing results with 4-5-year-olds and adults. In Experiment 2, a new, forced-choice immediate recognition memory test was implemented with the same children. On each trial, a card (photograph of a simple scene) was immediately replaced by a test card (identical view and either a closer or more wide-angle view) and participants indicated which one matched the original view. Error patterns supported boundary extension; identical photographs were more frequently rejected when the closer view was the original view, than vice versa. This asymmetry was not attributable to a selection bias (guessing tasks; Experiments 3-5). In Experiment 4, working memory load was increased by presenting more expansive views of more complex scenes. Again, children exhibited boundary extension, but now adults did not, unless stimulus duration was reduced to 5 s (limiting time to implement strategies; Experiment 5). We propose that like adults, children interpret photographs as views of places in the world; they extrapolate the anticipated continuation of the scene beyond the view and misattribute it to having been seen. Developmental differences in source attribution decision processes provide an explanation for the age-related differences observed.

Visual, haptic and bimodal scene perception: evidence for a unitary representation

Participants studied seven meaningful scene-regions bordered by removable boundaries (30s each). In Experiment 1 (N = 80) participants used visual or haptic exploration and then minutes later, reconstructed boundary position using the same or the alternate modality. Participants in all groups shifted boundary placement outward (boundary extension), but visual study yielded the greater error. Critically, this modality-specific difference in boundary extension transferred without cost in the cross-modal conditions, suggesting a functionally unitary scene representation. In Experiment 2 (N = 20), bimodal study led to boundary extension that did not differ from haptic exploration alone, suggesting that bimodal spatial memory was constrained by the more "conservative" haptic modality. In Experiment 3 (N = 20), as in picture studies, boundary memory was tested 30s after viewing each scene-region and as with pictures, boundary extension still occurred. Results suggest that scene representation is organized around an amodal spatial core that organizes bottom-up information from multiple modalities in combination with top-down expectations about the surrounding world.

Objects influence the shape of remembered views: examining global and local aspects of boundary extension

Boundary extension is a constructive memory error for views of scenes in which viewx tendto be remembered as more spatially expansive than they appeared. In seven experiments xwe examinedwhether local differences in boundary extension within views might exist by presenting par ticipants ith overhead views of single, elongated objects or object pairs on textured ground surfaces i:n whichobjects were oriented either vertically or horizontally. Memory for views' spatial expanse xwas testedwith either a view-recognition test or a border-adjustment test in which participants could ad just thespatial expanse of a test view using the computer's mouse. The border-adjustment test was uused tossess local boundary extension (primarily); the view-recognition test was used to assess paparticipants'emories for the overall spatial expanse of views (ie global boundary extension). Across exexperiments,n the border-adjustment tests specifically, participants showed more spatial expanse along the obobject'songer axis, in some cases restricting the view along the object's shorter axis. In addition, the rerecognition-t data revealed greater boundary extension for views with vertically oriented objects than for x.iviewsth horizontally oriented objects. Taken together, the results suggest that objects in scene x iviews canfect both local and global aspects of memory for spatial expanse of scene views.

An influence of extremal edges on boundary extension

Studies have shown that people consistently remember seeing more of a studied scene than was physically present (e.g., Intraub & Richardson Journal of Experimental Psychology: Learning, Memory, and Cognition, 15, 179-187, 1989). This scene memory error, known as boundary extension, has been suggested to occur due to an observer's failure to differentiate between the contributing sources of information, including the sensory input, amodal continuation beyond the view boundaries, and contextual associations with the main objects and depicted scene locations (Intraub, 2010). Here, "scenes" made of abstract shapes on random-dot backgrounds, previously shown to elicit boundary extension (McDunn, Siddiqui, & Brown Psychonomic Bulletin & Review, 21, 370-375, 2014), were compared with versions made with extremal edges (Palmer & Ghose Psychological Science, 19, 77-84, 2008) added to their borders, in order to examine how boundary extension is influenced when amodal continuation at the borders' view boundaries is manipulated in this way. Extremal edges were expected to reduce boundary extension as compared to the same scenes without them, because extremal edge boundaries explicitly indicate an image's end (i.e., they do not continue past the view boundary). A large and a small difference (16 % and 40 %) between the close and wide-angle views shown during the experiment were tested to examine the effects of both boundary extension and normalization with and without extremal edges. Images without extremal edges elicited typical boundary extension for the 16 % size change condition, whereas the 40 % condition showed signs of normalization. With extremal edges, a reduced amount of boundary extension occurred for the 16 % condition, and only normalization was found for the 40 % condition. Our findings support and highlight the importance of amodal continuation at the view boundaries as a component of boundary extension.